From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f47.google.com (mail-wm0-f47.google.com [74.125.82.47]) by dpdk.org (Postfix) with ESMTP id C67F6558B for ; Mon, 4 Jul 2016 03:14:21 +0200 (CEST) Received: by mail-wm0-f47.google.com with SMTP id r201so95817178wme.1 for ; Sun, 03 Jul 2016 18:14:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/oL+AdNDUPx3rMybaZq0akY25+0C3fii0YpBb5OQLb8=; b=k2nRIeSmLZi1RvHJAQaiOSHeBDcH6WGaxWQXFHNN1BwBQWXBo4YjZzZDUfjDILg73T /THDh45YHvUzQajQ80+acmc1KZuwUxq4PuQyF3yezHlEt5SJTGamjf9Kum89s1FqFeSI jKZwB66ffelB3zstz2+mlOucKVc/jTH/gZKz5u6Kh/2jd0u+Z0GhsrqbtizAWLH8NyM/ vrAj7tusgxwWtOA5KVYTMXx1tAyPF3hqIK0XnPvVdU79bgBnJanbFuxI9kNh0fhCFuDl rA+tA7bd8ah4ym8Ok8iaEA5bqMMOm6MvbaXsfCYwLNYukHcSfTFuSfNckUoiyQ493nk8 Gd8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/oL+AdNDUPx3rMybaZq0akY25+0C3fii0YpBb5OQLb8=; b=UBaNU0f5FydEBXBGKivyojCx1C8w9poSO/0YF5LsEyJpXYD7soX40xa6bZAB++Fmv+ SIxlefjOlkeq5FfbfrVnYiZ7+nJSsK4D38Ci91ff3JoW/jxjW4+saRwrwnJpEvCM4Pja Li9y+/3xCQE4cKA3GWh8s1BZRfzOu6VReFiE7qtT7UKxSnq+r9QDSJiI4Q0CZkad2vyP baU6FhGPPgC6zYQD7WQ7u9jwFnQe+nUSK7XZWTT90SeNCEe76LS/MYgSA2ClZGS6soVU UhtLsXIQ6+zZ+K8DHmFwY8Xf+afxQj4dB9o1X1vh5Ef/dJ2dYl1bAzjZQLqqLzJ5+56n q+rw== X-Gm-Message-State: ALyK8tKzjNIADbvlnUxHYDoKjxHuSyfu8vRwX085yhwpo0upBKjrErxrSmVYzCKQ4vJlTqSa X-Received: by 10.194.123.166 with SMTP id mb6mr8703061wjb.94.1467594861408; Sun, 03 Jul 2016 18:14:21 -0700 (PDT) Received: from XPS13.localdomain (184.203.134.77.rev.sfr.net. [77.134.203.184]) by smtp.gmail.com with ESMTPSA id l4sm5527127wjk.18.2016.07.03.18.14.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 03 Jul 2016 18:14:20 -0700 (PDT) From: Thomas Monjalon To: Neil Horman Cc: dev@dpdk.org, Panu Matilainen Date: Mon, 4 Jul 2016 03:14:05 +0200 Message-Id: <1467594845-3487-8-git-send-email-thomas.monjalon@6wind.com> X-Mailer: git-send-email 2.7.0 In-Reply-To: <1467594845-3487-1-git-send-email-thomas.monjalon@6wind.com> References: <1466189185-21952-1-git-send-email-nhorman@tuxdriver.com> <1467594845-3487-1-git-send-email-thomas.monjalon@6wind.com> Subject: [dpdk-dev] [PATCH v9 7/7] tools: query binaries for support information X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2016 01:14:22 -0000 From: Neil Horman This tool searches for the primer string PMD_INFO_STRING= in any ELF binary, and, if found parses the remainder of the string as a json encoded string, outputting the results in either a human readable or raw, script parseable format. Note that, in the case of dynamically linked applications, pmdinfo.py will scan for implicitly linked PMDs by searching the specified binaries .dynamic section for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH, LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that order. If a file is specified with no path, it is assumed to be a PMD DSO, and the LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it. Currently the tool can output data in 3 formats: a) raw, suitable for scripting, where the raw JSON strings are dumped out b) table format (default) where hex pci ids are dumped in a table format c) pretty, where a user supplied pci.ids file is used to print out vendor and device strings There is a dependency on pyelftools. The script is not yet compatible with Python 3. Signed-off-by: Neil Horman Acked-by: Panu Matilainen Acked-by: Remy Horton --- MAINTAINERS | 1 + lib/librte_eal/common/eal_common_options.c | 2 +- mk/rte.sdkinstall.mk | 2 + tools/dpdk-pmdinfo.py | 628 +++++++++++++++++++++++++++++ 4 files changed, 632 insertions(+), 1 deletion(-) create mode 100755 tools/dpdk-pmdinfo.py diff --git a/MAINTAINERS b/MAINTAINERS index 1a8a3b7..1e972f0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -71,6 +71,7 @@ F: scripts/validate-abi.sh Driver information M: Neil Horman F: buildtools/pmdinfogen/ +F: tools/dpdk-pmdinfo.py Environment Abstraction Layer diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 7e9f7b8..b562c8a 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -115,7 +115,7 @@ TAILQ_HEAD_INITIALIZER(solib_list); /* Default path of external loadable drivers */ static const char *default_solib_dir = RTE_EAL_PMD_PATH; -/* Stringified version of default solib path */ +/* Stringified version of default solib path used by dpdk-pmdinfo.py */ static const char dpdk_solib_path[] __attribute__((used)) = "DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH; diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk index 2b92157..76be308 100644 --- a/mk/rte.sdkinstall.mk +++ b/mk/rte.sdkinstall.mk @@ -126,6 +126,8 @@ install-runtime: $(Q)$(call rte_mkdir, $(DESTDIR)$(sbindir)) $(Q)$(call rte_symlink, $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \ $(DESTDIR)$(sbindir)/dpdk_nic_bind) + $(Q)$(call rte_symlink, $(DESTDIR)$(datadir)/tools/dpdk-pmdinfo.py, \ + $(DESTDIR)$(bindir)/dpdk-pmdinfo) install-kmod: ifneq ($(wildcard $O/kmod/*),) diff --git a/tools/dpdk-pmdinfo.py b/tools/dpdk-pmdinfo.py new file mode 100755 index 0000000..b8a9be2 --- /dev/null +++ b/tools/dpdk-pmdinfo.py @@ -0,0 +1,628 @@ +#!/usr/bin/python +# ------------------------------------------------------------------------- +# +# Utility to dump PMD_INFO_STRING support from an object file +# +# ------------------------------------------------------------------------- +import os +import sys +from optparse import OptionParser +import string +import json + +# For running from development directory. It should take precedence over the +# installed pyelftools. +sys.path.insert(0, '.') + + +from elftools import __version__ +from elftools.common.exceptions import ELFError +from elftools.common.py3compat import ( + ifilter, byte2int, bytes2str, itervalues, str2bytes) +from elftools.elf.elffile import ELFFile +from elftools.elf.dynamic import DynamicSection, DynamicSegment +from elftools.elf.enums import ENUM_D_TAG +from elftools.elf.segments import InterpSegment +from elftools.elf.sections import SymbolTableSection +from elftools.elf.gnuversions import ( + GNUVerSymSection, GNUVerDefSection, + GNUVerNeedSection, +) +from elftools.elf.relocation import RelocationSection +from elftools.elf.descriptions import ( + describe_ei_class, describe_ei_data, describe_ei_version, + describe_ei_osabi, describe_e_type, describe_e_machine, + describe_e_version_numeric, describe_p_type, describe_p_flags, + describe_sh_type, describe_sh_flags, + describe_symbol_type, describe_symbol_bind, describe_symbol_visibility, + describe_symbol_shndx, describe_reloc_type, describe_dyn_tag, + describe_ver_flags, +) +from elftools.elf.constants import E_FLAGS +from elftools.dwarf.dwarfinfo import DWARFInfo +from elftools.dwarf.descriptions import ( + describe_reg_name, describe_attr_value, set_global_machine_arch, + describe_CFI_instructions, describe_CFI_register_rule, + describe_CFI_CFA_rule, +) +from elftools.dwarf.constants import ( + DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file) +from elftools.dwarf.callframe import CIE, FDE + +raw_output = False +pcidb = None + +# =========================================== + + +class Vendor: + """ + Class for vendors. This is the top level class + for the devices belong to a specific vendor. + self.devices is the device dictionary + subdevices are in each device. + """ + + def __init__(self, vendorStr): + """ + Class initializes with the raw line from pci.ids + Parsing takes place inside __init__ + """ + self.ID = vendorStr.split()[0] + self.name = vendorStr.replace("%s " % self.ID, "").rstrip() + self.devices = {} + + def add_device(self, deviceStr): + """ + Adds a device to self.devices + takes the raw line from pci.ids + """ + s = deviceStr.strip() + devID = s.split()[0] + if devID in self.devices: + pass + else: + self.devices[devID] = Device(deviceStr) + + def report(self): + print self.ID, self.name + for id, dev in self.devices.items(): + dev.report() + + def find_device(self, devid): + # convert to a hex string and remove 0x + devid = hex(devid)[2:] + try: + return self.devices[devid] + except: + return Device("%s Unknown Device" % devid) + + +class Device: + + def __init__(self, deviceStr): + """ + Class for each device. + Each vendor has its own devices dictionary. + """ + s = deviceStr.strip() + self.ID = s.split()[0] + self.name = s.replace("%s " % self.ID, "") + self.subdevices = {} + + def report(self): + print "\t%s\t%s" % (self.ID, self.name) + for subID, subdev in self.subdevices.items(): + subdev.report() + + def add_sub_device(self, subDeviceStr): + """ + Adds a subvendor, subdevice to device. + Uses raw line from pci.ids + """ + s = subDeviceStr.strip() + spl = s.split() + subVendorID = spl[0] + subDeviceID = spl[1] + subDeviceName = s.split(" ")[-1] + devID = "%s:%s" % (subVendorID, subDeviceID) + self.subdevices[devID] = SubDevice( + subVendorID, subDeviceID, subDeviceName) + + def find_subid(self, subven, subdev): + subven = hex(subven)[2:] + subdev = hex(subdev)[2:] + devid = "%s:%s" % (subven, subdev) + + try: + return self.subdevices[devid] + except: + if (subven == "ffff" and subdev == "ffff"): + return SubDevice("ffff", "ffff", "(All Subdevices)") + else: + return SubDevice(subven, subdev, "(Unknown Subdevice)") + + +class SubDevice: + """ + Class for subdevices. + """ + + def __init__(self, vendor, device, name): + """ + Class initializes with vendorid, deviceid and name + """ + self.vendorID = vendor + self.deviceID = device + self.name = name + + def report(self): + print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name) + + +class PCIIds: + """ + Top class for all pci.ids entries. + All queries will be asked to this class. + PCIIds.vendors["0e11"].devices["0046"].\ + subdevices["0e11:4091"].name = "Smart Array 6i" + """ + + def __init__(self, filename): + """ + Prepares the directories. + Checks local data file. + Tries to load from local, if not found, downloads from web + """ + self.version = "" + self.date = "" + self.vendors = {} + self.contents = None + self.read_local(filename) + self.parse() + + def report_vendors(self): + """Reports the vendors + """ + for vid, v in self.vendors.items(): + print v.ID, v.name + + def report(self, vendor=None): + """ + Reports everything for all vendors or a specific vendor + PCIIds.report() reports everything + PCIIDs.report("0e11") reports only "Compaq Computer Corporation" + """ + if vendor is not None: + self.vendors[vendor].report() + else: + for vID, v in self.vendors.items(): + v.report() + + def find_vendor(self, vid): + # convert vid to a hex string and remove the 0x + vid = hex(vid)[2:] + + try: + return self.vendors[vid] + except: + return Vendor("%s Unknown Vendor" % (vid)) + + def find_date(self, content): + for l in content: + if l.find("Date:") > -1: + return l.split()[-2].replace("-", "") + return None + + def parse(self): + if len(self.contents) < 1: + print "data/%s-pci.ids not found" % self.date + else: + vendorID = "" + deviceID = "" + for l in self.contents: + if l[0] == "#": + continue + elif len(l.strip()) == 0: + continue + else: + if l.find("\t\t") == 0: + self.vendors[vendorID].devices[ + deviceID].add_sub_device(l) + elif l.find("\t") == 0: + deviceID = l.strip().split()[0] + self.vendors[vendorID].add_device(l) + else: + vendorID = l.split()[0] + self.vendors[vendorID] = Vendor(l) + + def read_local(self, filename): + """ + Reads the local file + """ + self.contents = open(filename).readlines() + self.date = self.find_date(self.contents) + + def load_local(self): + """ + Loads database from local. If there is no file, + it creates a new one from web + """ + self.date = idsfile[0].split("/")[1].split("-")[0] + self.read_local() + + +# ======================================= + +def search_file(filename, search_path): + """ Given a search path, find file with requested name """ + for path in string.split(search_path, ":"): + candidate = os.path.join(path, filename) + if os.path.exists(candidate): + return os.path.abspath(candidate) + return None + + +class ReadElf(object): + """ display_* methods are used to emit output into the output stream + """ + + def __init__(self, file, output): + """ file: + stream object with the ELF file to read + + output: + output stream to write to + """ + self.elffile = ELFFile(file) + self.output = output + + # Lazily initialized if a debug dump is requested + self._dwarfinfo = None + + self._versioninfo = None + + def _section_from_spec(self, spec): + """ Retrieve a section given a "spec" (either number or name). + Return None if no such section exists in the file. + """ + try: + num = int(spec) + if num < self.elffile.num_sections(): + return self.elffile.get_section(num) + else: + return None + except ValueError: + # Not a number. Must be a name then + return self.elffile.get_section_by_name(str2bytes(spec)) + + def pretty_print_pmdinfo(self, pmdinfo): + global pcidb + + for i in pmdinfo["pci_ids"]: + vendor = pcidb.find_vendor(i[0]) + device = vendor.find_device(i[1]) + subdev = device.find_subid(i[2], i[3]) + print("%s (%s) : %s (%s) %s" % + (vendor.name, vendor.ID, device.name, + device.ID, subdev.name)) + + def parse_pmd_info_string(self, mystring): + global raw_output + global pcidb + + optional_pmd_info = [{'id': 'params', 'tag': 'PMD PARAMETERS'}] + + i = mystring.index("=") + mystring = mystring[i + 2:] + pmdinfo = json.loads(mystring) + + if raw_output: + print(pmdinfo) + return + + print("PMD NAME: " + pmdinfo["name"]) + for i in optional_pmd_info: + try: + print("%s: %s" % (i['tag'], pmdinfo[i['id']])) + except KeyError as e: + continue + + if (len(pmdinfo["pci_ids"]) != 0): + print("PMD HW SUPPORT:") + if pcidb is not None: + self.pretty_print_pmdinfo(pmdinfo) + else: + print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE") + for i in pmdinfo["pci_ids"]: + print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % + (i[0], i[1], i[2], i[3])) + + print("") + + def display_pmd_info_strings(self, section_spec): + """ Display a strings dump of a section. section_spec is either a + section number or a name. + """ + section = self._section_from_spec(section_spec) + if section is None: + return + + data = section.data() + dataptr = 0 + + while dataptr < len(data): + while (dataptr < len(data) and + not (32 <= byte2int(data[dataptr]) <= 127)): + dataptr += 1 + + if dataptr >= len(data): + break + + endptr = dataptr + while endptr < len(data) and byte2int(data[endptr]) != 0: + endptr += 1 + + mystring = bytes2str(data[dataptr:endptr]) + rc = mystring.find("PMD_INFO_STRING") + if (rc != -1): + self.parse_pmd_info_string(mystring) + + dataptr = endptr + + def find_librte_eal(self, section): + for tag in section.iter_tags(): + if tag.entry.d_tag == 'DT_NEEDED': + if "librte_eal" in tag.needed: + return tag.needed + return None + + def search_for_autoload_path(self): + scanelf = self + scanfile = None + library = None + + section = self._section_from_spec(".dynamic") + try: + eallib = self.find_librte_eal(section) + if eallib is not None: + ldlibpath = os.environ.get('LD_LIBRARY_PATH') + if ldlibpath is None: + ldlibpath = "" + dtr = self.get_dt_runpath(section) + library = search_file(eallib, + dtr + ":" + ldlibpath + + ":/usr/lib64:/lib64:/usr/lib:/lib") + if library is None: + return (None, None) + if raw_output is False: + print("Scanning for autoload path in %s" % library) + scanfile = open(library, 'rb') + scanelf = ReadElf(scanfile, sys.stdout) + except AttributeError: + # Not a dynamic binary + pass + except ELFError: + scanfile.close() + return (None, None) + + section = scanelf._section_from_spec(".rodata") + if section is None: + if scanfile is not None: + scanfile.close() + return (None, None) + + data = section.data() + dataptr = 0 + + while dataptr < len(data): + while (dataptr < len(data) and + not (32 <= byte2int(data[dataptr]) <= 127)): + dataptr += 1 + + if dataptr >= len(data): + break + + endptr = dataptr + while endptr < len(data) and byte2int(data[endptr]) != 0: + endptr += 1 + + mystring = bytes2str(data[dataptr:endptr]) + rc = mystring.find("DPDK_PLUGIN_PATH") + if (rc != -1): + rc = mystring.find("=") + return (mystring[rc + 1:], library) + + dataptr = endptr + if scanfile is not None: + scanfile.close() + return (None, None) + + def get_dt_runpath(self, dynsec): + for tag in dynsec.iter_tags(): + if tag.entry.d_tag == 'DT_RUNPATH': + return tag.runpath + return "" + + def process_dt_needed_entries(self): + """ Look to see if there are any DT_NEEDED entries in the binary + And process those if there are + """ + global raw_output + runpath = "" + ldlibpath = os.environ.get('LD_LIBRARY_PATH') + if ldlibpath is None: + ldlibpath = "" + + dynsec = self._section_from_spec(".dynamic") + try: + runpath = self.get_dt_runpath(dynsec) + except AttributeError: + # dynsec is None, just return + return + + for tag in dynsec.iter_tags(): + if tag.entry.d_tag == 'DT_NEEDED': + rc = tag.needed.find("librte_pmd") + if (rc != -1): + library = search_file(tag.needed, + runpath + ":" + ldlibpath + + ":/usr/lib64:/lib64:/usr/lib:/lib") + if library is not None: + if raw_output is False: + print("Scanning %s for pmd information" % library) + with open(library, 'rb') as file: + try: + libelf = ReadElf(file, sys.stdout) + except ELFError as e: + print("%s is no an ELF file" % library) + continue + libelf.process_dt_needed_entries() + libelf.display_pmd_info_strings(".rodata") + file.close() + + +def scan_autoload_path(autoload_path): + global raw_output + + if os.path.exists(autoload_path) is False: + return + + try: + dirs = os.listdir(autoload_path) + except OSError as e: + # Couldn't read the directory, give up + return + + for d in dirs: + dpath = os.path.join(autoload_path, d) + if os.path.isdir(dpath): + scan_autoload_path(dpath) + if os.path.isfile(dpath): + try: + file = open(dpath, 'rb') + readelf = ReadElf(file, sys.stdout) + except ELFError as e: + # this is likely not an elf file, skip it + continue + except IOError as e: + # No permission to read the file, skip it + continue + + if raw_output is False: + print("Hw Support for library %s" % d) + readelf.display_pmd_info_strings(".rodata") + file.close() + + +def scan_for_autoload_pmds(dpdk_path): + """ + search the specified application or path for a pmd autoload path + then scan said path for pmds and report hw support + """ + global raw_output + + if (os.path.isfile(dpdk_path) is False): + if raw_output is False: + print("Must specify a file name") + return + + file = open(dpdk_path, 'rb') + try: + readelf = ReadElf(file, sys.stdout) + except ElfError as e: + if raw_output is False: + print("Unable to parse %s" % file) + return + + (autoload_path, scannedfile) = readelf.search_for_autoload_path() + if (autoload_path is None or autoload_path is ""): + if (raw_output is False): + print("No autoload path configured in %s" % dpdk_path) + return + if (raw_output is False): + if (scannedfile is None): + scannedfile = dpdk_path + print("Found autoload path %s in %s" % (autoload_path, scannedfile)) + + file.close() + if (raw_output is False): + print("Discovered Autoload HW Support:") + scan_autoload_path(autoload_path) + return + + +def main(stream=None): + global raw_output + global pcidb + + optparser = OptionParser( + usage='usage: %prog [-hrtp] [-d ', + description="Dump pmd hardware support info", + add_help_option=True, + prog='pmdinfo.py') + optparser.add_option('-r', '--raw', + action='store_true', dest='raw_output', + help='Dump raw json strings') + optparser.add_option("-d", "--pcidb", dest="pcifile", + help="specify a pci database " + "to get vendor names from", + default="/usr/share/hwdata/pci.ids", metavar="FILE") + optparser.add_option("-t", "--table", dest="tblout", + help="output information on hw support as a hex table", + action='store_true') + optparser.add_option("-p", "--plugindir", dest="pdir", + help="scan dpdk for autoload plugins", + action='store_true') + + options, args = optparser.parse_args() + + if options.raw_output: + raw_output = True + + if options.pcifile: + pcidb = PCIIds(options.pcifile) + if pcidb is None: + print("Pci DB file not found") + exit(1) + + if options.tblout: + options.pcifile = None + pcidb = None + + if (len(args) == 0): + optparser.print_usage() + exit(1) + + if options.pdir is True: + exit(scan_for_autoload_pmds(args[0])) + + ldlibpath = os.environ.get('LD_LIBRARY_PATH') + if (ldlibpath is None): + ldlibpath = "" + + if (os.path.exists(args[0]) is True): + myelffile = args[0] + else: + myelffile = search_file( + args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib") + + if (myelffile is None): + print("File not found") + sys.exit(1) + + with open(myelffile, 'rb') as file: + try: + readelf = ReadElf(file, sys.stdout) + readelf.process_dt_needed_entries() + readelf.display_pmd_info_strings(".rodata") + sys.exit(0) + + except ELFError as ex: + sys.stderr.write('ELF error: %s\n' % ex) + sys.exit(1) + + +# ------------------------------------------------------------------------- +if __name__ == '__main__': + main() -- 2.7.0