From: Neil Horman <nhorman@tuxdriver.com>
To: dev@dpdk.org
Cc: Bruce Richardson <bruce.richardson@intel.com>,
Thomas Monjalon <thomas.monjalon@6wind.com>,
Stephen Hemminger <stephen@networkplumber.org>,
Panu Matilainen <pmatilai@redhat.com>
Subject: Re: [dpdk-dev] [PATCHv6 0/7] Implement pmd hardware support exports
Date: Sat, 4 Jun 2016 20:20:58 -0400 [thread overview]
Message-ID: <20160605002058.GB8200@neilslaptop.think-freely.org> (raw)
In-Reply-To: <1464703068-2924-1-git-send-email-nhorman@tuxdriver.com>
On Tue, May 31, 2016 at 09:57:41AM -0400, Neil Horman wrote:
> Hey all-
> So heres attempt number 2 at a method for exporting PMD hardware support
> information. As we discussed previously, the consensus seems to be that pmd
> information should be:
>
> 1) Able to be interrogated on any ELF binary (application binary or individual
> DSO)
> 2) Equally functional on statically linked applications or on DSO's
> 3) Resilient to symbol stripping
> 4) Script friendly
> 5) Show kernel dependencies
> 6) List driver options
> 7) Show driver name
> 8) Offer human readable output
> 9) Show DPDK version
> 10) Show driver version
> 11) Allow for expansion
> 12) Not place additional build environment dependencies on an application
>
> I added item 12 myself, because I don't think its reasonable to require
> applications to use a specific linker script to get hardware support information
> (which precludes the use of a special .modinfo section like the kernel uses)
>
> However, we still can use some tricks from the kernel to make this work. In
> this approach, what I've done is the following:
>
> A) Modified the driver registration macro to also define a variable:
> this_pmd_name<n>= "name"
>
> Based on the unique name string pointed to by the above variable, we can
> query for an arbitrary number of other symbols following the pattern:
> <name>_<tag>
>
> Where tag is some well known identifier for information we wish to export
>
> B) Added a utility called pmdinfogen. This utility is not meant for general use,
> but rather is used by the dpdk build environment itself when compiling pmds.
> for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run. It
> searches for the symbols in (A), and using those, extracts the hardware support
> info, and module name from the object, using that to produce a new c file
> containing a single variable in the following format:
>
> static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";
>
> The <name> is arbitrary, as its static and not referenced. The relevant bit is
> the string value assigned to it. The <json> is a json encoded string of the
> extracted hardware support information pmdinfo found for the corresponding
> object. This C file is suitable for compilation and relocatable linking back
> into the parent object file. The result of this operation is that the object
> string table now contains a string that will not e removed by stripping, whos
> leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
> the symbol referring to it is stripped or not.
>
> C) Added a utilty called pmdinfo.py. This python script, searches the
> string table of the .rodata section of any provided ELF file looking for the
> PMD_DRIVER_INFO prefix on a string. When found, it will interpret the remainder
> of the string as json, and output the hardware support for that ELF file (if
> any).
>
>
> This approach ticks most of the above boxes:
> 1) Impervious to stripping
> 2) Works on static and dynamic binaries
> 3) Human and script friendly
> 4) Allows for expansion
>
> Because of (4) the other items should be pretty easy to implement, as its just a
> matter of modifying the macros to export the info, pmdinfo to encode it to json,
> and pmd_hw_support.py to read it. I'm not adding them now, as theres alot of
> rote work to do to get some of them in place (e.g. DPDK has no current global
> VERSION macro, drivers don't have a consistent version scheme, command line
> strings have to be built for each driver, etc). But once this is accepted,
> those items can be done as time allows and should be fairly easy to implement.
>
> Change Notes:
> v2)
> * Made the export macros a bit easier to expand
> * Added a macro to optionally export command line strings
> * Renamed utilties to be more appropriate
> (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
> * Added search capabilities to pmdinfo.py so that we search for libraries
> linked using DT_NEEDINFO entries. We search DT_RUNPATH, LD_LIBRARY_PATH,
> /usr/lib and /lib
> * Added serch abilities if the specified binary to pmdinfo isn't found, we
> search LD_LIBRARY_PATH, /usr/lib and /lib
> * Added an option to pmdinfo.py to pretty-print hardware support info using the
> pci.ids database
>
> v3)
> * Made table mode printing the default mode
> * Added a default path to the pci.ids file
> * Modifed pmdinfo to use python rather than python3
> * Cleaned up some changelog entries
> * Added an export for RTE_EAL_PMD_PATH
> * Added a plugin option to pmdinfo to scan for autoloaded DSO's
>
> v4)
> * Modified the operation of the -p option. As much as I don't like implying
> that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> time seeing how we can avoid specifying the application file to scan for the
> autoload directory. Without it we can't determine which library the user means
> in a multiversion installation
> * Cleaned up the help text
> * Added a rule for an install target for pmdinfo
> * Guarded against some tracebacks in pmdinfo
> * Use DT_NEEDED entries to get versioned libraries in -p mode
> * Fixed traceback that occurs on lack of input arguments
> * Fixed some erroneous macro usage in drivers that aren't in the default build
>
> v5)
> * Added a dpdk- prefix to pmdinfo symlink
> * Renamed build announcement to PMDINFOGEN/BUILD
> * Removed erroneous true statement from makefile
> * Removed duplicate rte.hostapp.mk makefile
> * Fixed some whitespace
> * Whitespace fixups
> * Fixed makefile if; then style
> * Renamed module string C file
> * Removed duplicate rte_pci_id definition
> * Clarified macro names
> * Removed PMD type attribute
> * Fixed tools usage for 32 bit arches
> * Removed some unused code
> * Added a few comments
>
> v6)
> * Added some programming guide documentation
> * Reformatted python script with pep8
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
>
>
>
>
>
>
>
>
>
>
Ping, feedback here?
Neil
next prev parent reply other threads:[~2016-06-05 0:21 UTC|newest]
Thread overview: 166+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-16 20:41 [dpdk-dev] [PATCH 0/4] " Neil Horman
2016-05-16 20:41 ` [dpdk-dev] [PATCH 1/4] pmdinfo: Add buildtools and pmdinfo utility Neil Horman
2016-05-16 20:41 ` [dpdk-dev] [PATCH 2/4] drivers: Update driver registration macro usage Neil Horman
2016-05-16 20:41 ` [dpdk-dev] [PATCH 3/4] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-16 20:41 ` [dpdk-dev] [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information Neil Horman
2016-05-18 11:48 ` Panu Matilainen
2016-05-18 12:03 ` Neil Horman
2016-05-18 12:48 ` Panu Matilainen
2016-05-18 13:48 ` Neil Horman
2016-05-19 6:08 ` Panu Matilainen
2016-05-19 13:26 ` Neil Horman
2016-05-20 7:30 ` Panu Matilainen
2016-05-20 14:06 ` Neil Horman
2016-05-23 11:56 ` Panu Matilainen
2016-05-23 13:55 ` Neil Horman
2016-05-24 6:15 ` Panu Matilainen
2016-05-24 14:55 ` Neil Horman
2016-05-18 12:38 ` Thomas Monjalon
2016-05-18 13:09 ` Panu Matilainen
2016-05-18 13:26 ` Thomas Monjalon
2016-05-18 13:54 ` Neil Horman
2016-05-18 21:08 ` [dpdk-dev] [PATCHv2 0/4] Implement pmd hardware support exports Neil Horman
2016-05-18 21:08 ` [dpdk-dev] [PATCHv2 1/4] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-05-19 7:51 ` Panu Matilainen
2016-05-19 12:00 ` Neil Horman
2016-05-18 21:08 ` [dpdk-dev] [PATCHv2 2/4] drivers: Update driver registration macro usage Neil Horman
2016-05-19 7:58 ` Panu Matilainen
2016-05-19 10:45 ` Neil Horman
2016-05-19 10:51 ` [dpdk-dev] [dpdk-dev, PATCHv2, " Jan Viktorin
[not found] ` <20160519124650.060aa09a@pcviktorin.fit.vutbr.cz>
2016-05-19 11:40 ` Neil Horman
2016-05-18 21:08 ` [dpdk-dev] [PATCHv2 3/4] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-18 21:08 ` [dpdk-dev] [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-05-19 9:02 ` Panu Matilainen
2016-05-19 12:00 ` Neil Horman
2016-05-20 5:22 ` Panu Matilainen
2016-05-20 8:55 ` Thomas Monjalon
2016-05-20 9:12 ` Panu Matilainen
2016-05-20 14:22 ` Neil Horman
2016-05-20 14:20 ` Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 0/5] Implement pmd hardware support exports Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 1/5] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 2/5] drivers: Update driver registration macro usage Neil Horman
2016-05-24 6:57 ` Panu Matilainen
2016-05-24 13:21 ` Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 3/5] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 4/5] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-20 17:24 ` [dpdk-dev] [PATCHv3 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-05-24 7:41 ` Panu Matilainen
2016-05-24 15:17 ` Neil Horman
2016-05-24 8:34 ` Panu Matilainen
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 0/5] Implement pmd hardware support exports Neil Horman
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-05-25 13:21 ` Thomas Monjalon
2016-05-25 17:22 ` Neil Horman
2016-05-25 17:39 ` Thomas Monjalon
2016-05-25 19:13 ` Neil Horman
2016-05-25 19:39 ` Thomas Monjalon
2016-05-25 19:57 ` Neil Horman
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 2/5] drivers: Update driver registration macro usage Neil Horman
2016-05-25 16:20 ` Thomas Monjalon
2016-05-25 17:35 ` Neil Horman
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 3/5] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-25 17:08 ` Thomas Monjalon
2016-05-25 17:40 ` Neil Horman
2016-05-25 18:56 ` Thomas Monjalon
2016-05-25 19:43 ` Neil Horman
2016-05-25 20:04 ` Thomas Monjalon
2016-05-25 20:16 ` Neil Horman
2016-05-24 19:41 ` [dpdk-dev] [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-05-25 17:22 ` Thomas Monjalon
2016-05-25 17:47 ` Neil Horman
2016-05-25 18:58 ` Thomas Monjalon
2016-05-27 9:16 ` Panu Matilainen
2016-05-27 11:35 ` Neil Horman
2016-05-27 12:46 ` Panu Matilainen
2016-05-25 8:32 ` [dpdk-dev] [PATCHv4 0/5] Implement pmd hardware support exports Panu Matilainen
2016-05-25 11:27 ` Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 0/6] " Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 1/6] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 2/6] drivers: Update driver registration macro usage Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 3/6] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 4/6] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-05-27 14:38 ` Mcnamara, John
2016-05-27 19:56 ` Neil Horman
2016-05-26 17:17 ` [dpdk-dev] [PATCHv5 6/6] remove rte.hostapp.mk Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 0/7] Implement pmd hardware support exports Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-06-07 9:57 ` Thomas Monjalon
2016-06-07 12:04 ` Neil Horman
2016-06-07 12:53 ` Thomas Monjalon
2016-06-07 13:03 ` Neil Horman
2016-06-07 13:24 ` Thomas Monjalon
2016-06-07 13:49 ` Neil Horman
2016-06-07 14:09 ` Thomas Monjalon
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 2/7] drivers: Update driver registration macro usage Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 3/7] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 4/7] Makefile: Do post processing on objects that register a driver Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 5/7] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 6/7] remove rte.hostapp.mk Neil Horman
2016-05-31 13:57 ` [dpdk-dev] [PATCHv6 7/7] doc: Add prog_guide section documenting pmdinfo script Neil Horman
2016-06-08 17:14 ` Mcnamara, John
2016-06-09 17:31 ` Neil Horman
2016-06-05 0:20 ` Neil Horman [this message]
2016-06-07 9:34 ` [dpdk-dev] [PATCHv6 0/7] Implement pmd hardware support exports Thomas Monjalon
2016-06-07 12:08 ` Neil Horman
2016-06-07 12:27 ` Thomas Monjalon
2016-06-09 17:46 ` [dpdk-dev] [PATCHv7 0/6] " Neil Horman
2016-06-09 17:46 ` [dpdk-dev] [PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-06-16 12:29 ` Panu Matilainen
2016-06-16 13:33 ` Neil Horman
2016-06-16 14:06 ` Panu Matilainen
2016-06-16 14:41 ` Neil Horman
2016-06-09 17:46 ` [dpdk-dev] [PATCHv7 2/6] drivers: Update driver registration macro usage Neil Horman
2016-06-09 17:46 ` [dpdk-dev] [PATCHv7 3/6] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-06-09 17:46 ` [dpdk-dev] [PATCHv7 4/6] Makefile: Do post processing on objects that register a driver Neil Horman
2016-06-09 17:47 ` [dpdk-dev] [PATCHv7 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-06-16 12:32 ` Panu Matilainen
2016-06-09 17:47 ` [dpdk-dev] [PATCHv7 6/6] doc: Add prog_guide section documenting pmdinfo script Neil Horman
2016-06-09 19:55 ` Mcnamara, John
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 0/6] Implement pmd hardware support exports Neil Horman
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 1/6] pmdinfogen: Add buildtools and pmdinfogen utility Neil Horman
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 2/6] drivers: Update driver registration macro usage Neil Horman
2016-07-07 11:00 ` De Lara Guarch, Pablo
2016-07-07 11:39 ` Neil Horman
2016-07-07 15:37 ` [dpdk-dev] [PATCH] crypto: normalize cryptodev pmd names with macros Neil Horman
2016-07-08 9:09 ` De Lara Guarch, Pablo
2016-07-08 12:17 ` Neil Horman
2016-07-08 12:40 ` Thomas Monjalon
2016-07-08 13:42 ` Neil Horman
2016-07-08 19:03 ` Mcnamara, John
2016-07-09 13:34 ` Neil Horman
2016-07-09 16:04 ` Mcnamara, John
2016-07-08 14:00 ` De Lara Guarch, Pablo
2016-07-08 14:14 ` Neil Horman
2016-07-08 10:03 ` Thomas Monjalon
2016-07-08 16:34 ` [dpdk-dev] [PATCH v2] " Pablo de Lara
2016-07-08 16:46 ` [dpdk-dev] [PATCH v3] " Pablo de Lara
2016-07-08 17:14 ` Thomas Monjalon
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 3/6] eal: Add an export symbol to expose the autoload path to external tools Neil Horman
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 4/6] Makefile: Do post processing on objects that register a driver Neil Horman
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information Neil Horman
2016-06-29 15:12 ` Remy Horton
2016-06-29 16:18 ` Neil Horman
2016-06-30 7:45 ` Remy Horton
2016-06-17 18:46 ` [dpdk-dev] [PATCHv8 6/6] doc: Add prog_guide section documenting pmdinfo script Neil Horman
2016-06-29 9:18 ` [dpdk-dev] [PATCHv8 0/6] Implement pmd hardware support exports Remy Horton
2016-06-29 11:31 ` Neil Horman
2016-06-30 7:45 ` Remy Horton
2016-07-06 21:21 ` Thomas Monjalon
2016-07-04 1:13 ` [dpdk-dev] [PATCH v9 0/7] export PMD infos Thomas Monjalon
2016-07-04 1:13 ` [dpdk-dev] [PATCH v9 1/7] drivers: export infos as string symbols Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 2/7] mk: remove recipe for tool library Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 3/7] mk: refresh recipe for any host application Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 4/7] pmdinfogen: parse driver to generate code to export Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 5/7] mk: link infos generated by pmdinfogen Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 6/7] eal: export default plugin path to external tools Thomas Monjalon
2016-07-04 1:14 ` [dpdk-dev] [PATCH v9 7/7] tools: query binaries for support information Thomas Monjalon
2016-07-04 12:34 ` [dpdk-dev] [PATCH v9 0/7] export PMD infos Neil Horman
2016-07-04 13:10 ` Thomas Monjalon
2016-07-04 16:41 ` Neil Horman
2016-07-04 20:10 ` Thomas Monjalon
2016-07-05 20:08 ` Neil Horman
2016-07-06 15:33 ` Thomas Monjalon
2016-07-04 15:22 ` Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160605002058.GB8200@neilslaptop.think-freely.org \
--to=nhorman@tuxdriver.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=pmatilai@redhat.com \
--cc=stephen@networkplumber.org \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).