DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [RFC] Yet another option for DPDK options
@ 2016-06-01 15:00 Wiles, Keith
  2016-06-01 15:46 ` Matthew Hall
                   ` (4 more replies)
  0 siblings, 5 replies; 49+ messages in thread
From: Wiles, Keith @ 2016-06-01 15:00 UTC (permalink / raw)
  To: Yuanhan Liu, Thomas Monjalon
  Cc: dev, Richardson, Bruce, Tan, Jianfeng, Stephen Hemminger,
	Christian Ehrhardt, Panu Matilainen, Olivier Matz

Started from the link below, but did not want to highjack the thread.
http://dpdk.org/ml/archives/dev/2016-June/040021.html

I was thinking about this problem from a user perspective and command line options are very difficult to manage specifically when you have a large number of options as we have in dpdk. I see all of these options as a type of database of information for the DPDK and the application, because the application command line options are also getting very complex as well.

I have been looking at a number of different options here and the direction I was thinking was using a file for the options and configurations with the data in a clean format. It could have been a INI file or JSON or XML, but they all seem to have some problems I do not like. The INI file is too flat and I wanted a hierarchy in the data, the JSON data is similar and XML is just hard to read. I wanted to be able to manage multiple applications and possible system the DPDK/app runs. The problem with the above formats is they are just data and not easy to make decisions about the system and applications at runtime.

If the “database” of information could be queried by the EAL, drivers and application then we do not need to try and create a complex command line. It would be nice to execute a DPDK applications like this:

./some_dpdk_app –config-file dpdk-config-filename

The dpdk-config-filename could contain a lot of information and be able to startup multiple different applications. The dpdk-config-file could also include other config files to complete the configuration. The format of the data in the config file needs to be readable, but allow the user to put in new options, needs to be hierarchical in nature and have some simple functions to execute if required.

The solution I was thinking is the file information is really just a fragment of a scripting language, which the DPDK application contains this scripting language interpreter. I was looking at using Lua lua.org as the scripting language interpreter it is small and easy to understand. Python and others are very big and require a lot of resources and Lua requires very few system resources. Also I did not want to have to write a parser (lex/yacc). The other nice feature of Lua is you can create a sandbox for the code to run in and limit the type of system resources and APIs that can be accessed by the application and configuration. Lua can be trimmed down to a fairly small size and builds on just about any system or we can just install Lua on the system without changes from a rpm or deb.

I use Lua in pktgen at this time and the interface between ‘C’ and Lua is very simple and easy. Currently I include Lua in Pktgen, but I could have just used a system library.

The data in the config file can be data statements along with some limited code to make some data changes at run time without having to modify the real application. Here is a simple config file I wrote: Some of the options do not make sense to be in the file at the same time, but wanted to see all of the options. The mk_lcore_list() and mk_coremap() are just Lua functions we can preload to help convert the simple strings into real data in this case tables of information. The application could be something like pktgen = { map = { … }, more_options = 1, } this allows the same file to possible contain many application configurations. Needs a bit more work.

dpdk_default = {
    lcore_mask = 0xFF00,
    lcore_list = mk_lcore_list("0-7", 10, "14-16"),
    coremap = mk_coremap("(0-7)@0,10,(14-16)@1"),
    master_lcore = 1,
    log_level = 7,
    ranks = 4,
    channels = 2,
    memory = 512,
    socket_mem = { 512, 512 },
    huge_dir = "/mnt/huge",
    base_virtaddr = 0,
    create_uio_dev = true,
    vfio_intr = "legacy",
    xen_dom0 = false,
    proc_type = "auto",
    pci_blacklist = {
        "08:00.0",
        "08:00.1",
        "09:00.0",
        "09:00.1",
        "83:00.1",
        "87:00.0",
        "87:00.1",
        "89:00.0",
        "89:00.1"
    },
    pci_whitelist = {
    },
    vdev = {
        eth_pcap0 = { iface = "eth2" },
        eth_pcap1 = { iface = "eth3" },
    },
    driver = { },
    syslog = true,
    vmware_tsc_map = false,
    file_prefix = "pg",
    huge_unlink = true,
    no_huge = false,
    no_pci = false,
    no_hpet = false,
    no_shconf = false,
}

pktgen_data = {
   map = { … },
   more-data = 1,
}

The EAL, driver, application, … would query an API to access the data and the application can change his options quickly without modifying the code.

Anyway comments are welcome.
 
Regards,
Keith






^ permalink raw reply	[flat|nested] 49+ messages in thread

end of thread, other threads:[~2016-06-05  0:20 UTC | newest]

Thread overview: 49+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-01 15:00 [dpdk-dev] [RFC] Yet another option for DPDK options Wiles, Keith
2016-06-01 15:46 ` Matthew Hall
2016-06-01 16:08   ` Wiles, Keith
2016-06-01 15:58 ` Jay Rolette
2016-06-01 16:18   ` Bruce Richardson
2016-06-01 16:21     ` Arnon Warshavsky
2016-06-01 18:13     ` Wiles, Keith
2016-06-01 18:31     ` Stephen Hemminger
2016-06-03 10:07     ` Yerden Zhumabekov
2016-06-01 18:51 ` Thomas Monjalon
2016-06-02  9:19   ` Marc
2016-06-02  7:56 ` Yuanhan Liu
2016-06-02 10:41 ` Neil Horman
2016-06-02 13:19   ` Thomas Monjalon
2016-06-02 13:53     ` Wiles, Keith
2016-06-02 17:11       ` Neil Horman
2016-06-02 19:33         ` Wiles, Keith
2016-06-02 19:41         ` Wiles, Keith
2016-06-02 20:08           ` Neil Horman
2016-06-02 20:53             ` Matthew Hall
2016-06-02 22:34               ` Neil Horman
2016-06-03  2:17                 ` Matthew Hall
2016-06-03  9:57                   ` Bruce Richardson
2016-06-03 10:06                     ` Bruce Richardson
2016-06-03 12:03                   ` Neil Horman
2016-06-03 10:29             ` Bruce Richardson
2016-06-03 11:01               ` Bruce Richardson
2016-06-03 11:50                 ` Neil Horman
2016-06-03 12:01                   ` Arnon Warshavsky
2016-06-03 12:53                     ` Panu Matilainen
2016-06-03 14:31                       ` Arnon Warshavsky
2016-06-03 16:04                         ` Wiles, Keith
2016-06-03 16:10                           ` Wiles, Keith
2016-06-03 17:44                           ` Neil Horman
2016-06-03 18:29                             ` Wiles, Keith
2016-06-03 18:38                               ` Neil Horman
2016-06-03 18:52                                 ` Arnon Warshavsky
2016-06-03 19:00                                   ` Wiles, Keith
2016-06-03 19:07                                     ` Wiles, Keith
2016-06-03 19:18                                       ` Neil Horman
2016-06-03 19:23                                         ` Wiles, Keith
2016-06-03 19:28                                           ` Arnon Warshavsky
2016-06-03 21:42                                           ` Matthew Hall
2016-06-03 21:41                                         ` Matthew Hall
2016-06-05  0:19                                           ` Neil Horman
2016-06-03 21:40                                       ` Matthew Hall
2016-06-03 21:38                                   ` Matthew Hall
2016-06-03 12:14                   ` Panu Matilainen
2016-06-02 20:51           ` Matthew Hall

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).