Re: [dpdk-users] Problems linking DPDK-20.11 libraries to another program

["Templin (US), Fred L" <Fred.L.Templin@boeing.com>]

All, I just noticed that the "dpdk-helloworld" binary was massively large
because it was built with the "static" option. When I build it as "shared",
the resulting binary is appropriately small. However, when I build it as
"shared" I get the same failure case that I describe for what I am getting
out of my testprog below.

I would greatly prefer to link DPDK into my testprog as "shared" and not
"static". But, if I link it as "shared" is there some sort of pre-initialization
sequence I need to execute before calling "rte_eal_init()"? I would be
happy to be pointed to the appropriate documentation.

Thanks - Fred

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Templin (US), Fred L
> Sent: Friday, April 16, 2021 12:32 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Problems linking DPDK-20.11 libraries to another program
> 
> Hello, I am at an impasse trying to link DPDK-20.11 libraries with another program.
> The other program (call it "testprog") is a large C source code application that uses
> standard linux open source build and compile facilities, including Makefiles, gcc,
> configure, etc. It is built from about 190 *.[ch] files, with one defining a single
> main() for the entire application.
> 
> I read the DPDK documentation and tried to link the libraries in using the
> "pkg-config" facility per the recommendations, but I was unable to get pkg-config
> to execute out of the "testprog" Makefile. So, I issued the following commands to
> obtain the results of running pkg-config:
> 
> fltemplin@aer-dtn:~$ pkg-config --cflags libdpdk
> fltemplin@aer-dtn:~$ pkg-config --libs libdpdk
> 
> I then added the results of the commands to the "testprog" Makefile by adding
> the following lines:
> 
> CFLAGS += -include rte_config.h -march=native -I/usr/local/include
> LDFLAGS += -L/usr/local/lib/x86_64-linux-gnu -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -
> lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -
> lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -
> lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer
> -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal
> -lrte_telemetry -lrte_kvargs
> 
> Next, I modified the DPDK "helloworld" example main.c to link it into "testprog" by
> changing the name to "hello.c" and changing the "main()" declaration to a function
> call called: "testprog_hello()" which I then called from one of my other "testprog"
> C modules. The diffs for how I modified the helloworld code appear at the end of
> this message.
> 
> I was then able to run "make" in the "testprog" directory, and the build compiled
> and appeared to link everything just fine. However, it appears that the resulting
> "tesprog" binary only increased in size by about 4% over the size it normally builds
> to without the DPDK libraries. This seems at odds with the fact that the DPDK
> "dpdk-helloworld" binary is massively large in comparison especially considering
> that the helloworld main.c is only 47 lines of C code! The sizes of the binaries
> are shown below:
> 
> fltemplin@aer-dtn:~$ cd ~/src/testprog/build
> fltemplin@aer-dtn:~/src/testprog/build$ ls -l testprog*
> -rwxr-xr-x 1 fltemplin fltemplin 4991800 Apr 16 10:25 testprog.nodpdk
> -rwxr-xr-x 1 fltemplin fltemplin 5186816 Apr 16 10:32 testprog.withdpdk
> fltemplin@aer-dtn~/src/testpog/build$: cd ~/src/DPDK/dpdk-20.11/build/examples/
> fltemplin@aer-dtn:~/src/DPDK/dpdk-20.11/build/examples$ ls -l dpdk-helloworld
> -rwxr-xr-x 1 fltemplin fltemplin 17834608 Apr 14 14:30 dpdk-helloworld
> 
> The next question then came to whether my combined testprog/dpdk code
> would run the same as the native DPDK helloworld example. I ultimately want
> to set up a NULL network interface when I call "rte_eal_init (argc, argv)" and
> so I tried issuing the following invocation of dpdk-helloworld:
> 
> fltemplin@aer-dtn:~$ sudo ./dpdk-helloworld --vdev net_null0
> EAL: Detected 1 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL:   Invalid NUMA socket, default to 0
> EAL:   Invalid NUMA socket, default to 0
> EAL: No legacy callbacks, legacy socket not created
> hello from core 0
> 
> So, as you can see, dpdk-helloworld passes the "--vnet" and "net_null0" arguments
> to "rte_eal_init()", which recognizes the arguments and returns success. But, when
> I call "rte_eal_init()" out of my testprog using the same arguments the DPDK service
> returns the following:
> 
> fltemplin@aer-dtn:~$ sudo ./testprog
> EAL: Detected 1 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: failed to parse device "net_null0"
> EAL: Unable to parse device 'net_null0'
> PANIC in testprog_hello():
> Cannot init EAL
> 
> This appears to me that DPDK is failing to initialize basic data structures
> when run out of my testprog that are initialized fine when run out of the
> dpdk-helloworld binary. That the code compiles and links is encouraging,
> but the fact that the binary size for my combined testprog/helloworld is
> significantly smaller than the size of dpdk-helloworld alone seems very
> suspicious. I feel I must be missing something very basic - does anyone
> recognize these conditions and have insights about how to fix?
> 
> Thanks - Fred
> 
> As previously mentioned, see below for my source code changes to link
> helloworld into testprog:
> 
> *** ~/src/DPDK/dpdk-20.11/examples/helloworld/main.c	2021-04-14 13:59:16.224073175 -0700
> --- ~/src/testprog/src/hello.c	2021-04-16 09:46:42.142210166 -0700
> ***************
> *** 2,7 ****
> --- 2,14 ----
>    * Copyright(c) 2010-2014 Intel Corporation
>    */
> 
> + #ifdef HAVE_CONFIG_H
> + #include "config.h"
> + #elif defined(_MSC_VER)
> + #include "config-msvc.h"
> + #endif
> +
> + #ifdef ENABLE_DPDK_TEST
>   #include <stdio.h>
>   #include <string.h>
>   #include <stdint.h>
> ***************
> *** 25,32 ****
> --- 32,47 ----
>   }
> 
>   int
> + #ifdef ENABLE_DPDK_TEST
> + testprog_hello()
> + {
> + 	char *args[] = { "openvpn", "--vdev", "net_null0" };
> + 	char **argv = args;
> + 	int argc = 3;
> + #else
>   main(int argc, char **argv)
>   {
> + #endif
>   	int ret;
>   	unsigned lcore_id;
> 
> ***************
> *** 45,47 ****
> --- 60,63 ----
>   	rte_eal_mp_wait_lcore();
>   	return 0;
>   }
> + #endif