From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 2C799A00C3;
	Tue, 20 Sep 2022 19:50:16 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 20EB140DFB;
	Tue, 20 Sep 2022 19:50:16 +0200 (CEST)
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on2070.outbound.protection.outlook.com [40.107.92.70])
 by mails.dpdk.org (Postfix) with ESMTP id 31A5B4069B
 for <dev@dpdk.org>; Tue, 20 Sep 2022 19:50:14 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=O7woZNpNhF7xn+Nk3iwDLdJhWaC3FIuIoVaTtHPyiGseu7CVNJFf0Vbm2ocVcami3drGoEUij1Gtrzu0UwvZxtW97VyCw086BDjXjSVPrRP1i/LyhlUdy9lnxM/F4REAJ160CGLk3nhuowRP8KLwTOHCiItMb7WMLbNZeAJVyfmYjXA85gBf9IaH21ZxDUIs48JJUfSL0xxyj/47XmQBsJpyiS4oLlvVR49NnbxvK87bi48vniwrSPAN9AQwCzJetVDYuODYMhaF9cvVt01+orfcC6J1rMQSTryo69nuRJ7r0ucgZRuMpZ+z3bS0oZOM7HNQ5L9xFHHAb+jCgkPhnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xwn7x+MR2QJVLIl6yKr0a97UbJNSr4PxdiQ8BNrfbtU=;
 b=TYicEJhk541TPKhvX5sl92N4lnKZ2hTB8FiHhCEZD3RLReHODCxgKLc8och18jBdcVPFYbjCfOfUrqE0c/4Y/kqJj50B9RtL2CJEKiX3bskz+Wt8OhMgs2Q+Ysv8J1OW3l1fQm2xVJ228NYU0PtL5FCQfPZz1SLd7Lyz/+oUm38gevb1SVd36MNLmgVR8OxTTdcU5hVoZ3npyiG4IA0hUevRGwxkKvcoxfMcQGFkO7iniv+q4JKTXjiip2jrQoShGzEOkoZ3nJM18ZSUjjhm4JgOJ2jtOwIPybT9PY3HafSDia1SoIqknDahOsm/2FLUoPNb2NGBuK1UUBeQpV+oQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.80.198) smtp.rcpttodomain=intel.com smtp.mailfrom=xilinx.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xwn7x+MR2QJVLIl6yKr0a97UbJNSr4PxdiQ8BNrfbtU=;
 b=r/sVFWsUVWhea4624Xo6KDK9vGlmmeGxS1yTPlcbwg2V0yJsUxZZi3jnmwLsCnpX4rAuiXQyeEuArszZ0DBVUYUK+ryLWS4b3oR/a2eDq7oTpJjLQzXEK72LV+Kuy/Oqnvns7Bs7Ptfk966rqWAXL0ZbGAHB+wwtFpQ1TyE3lZ8=
Received: from SN4PR0501CA0072.namprd05.prod.outlook.com
 (2603:10b6:803:41::49) by PH0PR02MB7432.namprd02.prod.outlook.com
 (2603:10b6:510:15::5) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.21; Tue, 20 Sep
 2022 17:50:11 +0000
Received: from SN1NAM02FT0009.eop-nam02.prod.protection.outlook.com
 (2603:10b6:803:41:cafe::c4) by SN4PR0501CA0072.outlook.office365.com
 (2603:10b6:803:41::49) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.6 via Frontend
 Transport; Tue, 20 Sep 2022 17:50:11 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198)
 smtp.mailfrom=xilinx.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.80.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.80.198; helo=xir-pvapexch01.xlnx.xilinx.com; pr=C
Received: from xir-pvapexch01.xlnx.xilinx.com (149.199.80.198) by
 SN1NAM02FT0009.mail.protection.outlook.com (10.97.4.92) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.5632.12 via Frontend Transport; Tue, 20 Sep 2022 17:50:11 +0000
Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by
 xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2375.24; Tue, 20 Sep 2022 18:50:10 +0100
Received: from smtp.xilinx.com (172.21.105.197) by
 xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id
 15.1.2375.24 via Frontend Transport; Tue, 20 Sep 2022 18:50:10 +0100
Envelope-to: john.mcnamara@intel.com, thomas@monjalon.net,
 olivier.matz@6wind.com, bruce.richardson@intel.com,
 rjarry@redhat.com, dev@dpdk.org
Received: from [10.71.194.74] (port=53560)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <ferruh.yigit@xilinx.com>)
 id 1oahNq-0007AI-2O; Tue, 20 Sep 2022 18:50:10 +0100
Message-ID: <12f168d2-1da8-7f90-b042-c91d25031467@xilinx.com>
Date: Tue, 20 Sep 2022 18:50:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.2
Subject: Re: [PATCH v4] usertools: rewrite pmdinfo
Content-Language: en-US
From: Ferruh Yigit <ferruh.yigit@xilinx.com>
To: "Mcnamara, John" <john.mcnamara@intel.com>, Thomas Monjalon
 <thomas@monjalon.net>
CC: Olivier Matz <olivier.matz@6wind.com>, Bruce Richardson
 <bruce.richardson@intel.com>, Robin Jarry <rjarry@redhat.com>, <dev@dpdk.org>
References: <20220913105811.84384-1-robin@jarry.cc>
 <20220920104212.476889-1-rjarry@redhat.com>
 <b17b105a-c288-e0c2-f3e2-3442364523b1@xilinx.com>
In-Reply-To: <b17b105a-c288-e0c2-f3e2-3442364523b1@xilinx.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1NAM02FT0009:EE_|PH0PR02MB7432:EE_
X-MS-Office365-Filtering-Correlation-Id: a9af0369-d9a0-4e9d-db03-08da9b3090c0
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: p3PeZ+4HUBAUXBMllDV2gKJiOXLi2xU5/EPXNQBk6OO+ANyVJFIE50SWj8NrWSTKZb5Reh4OJZYP6bWDqNU9fl3bxEEw3/64PrkwcCtSUv33Dhi1jrEjBEAnuUXAG5l3xhWMZIBlnzXOmH+rguuwpkE2lqRc9f2IyWsubdHh/40RyQif9Zz4yE47i05B9GBkqdHpZJkDf8mpmWD+bik7nAlrsC5i+f7tJX39A5lSVkekMaaTXwx91SNa3/imuSeX+lxLhuMcm1xY4yeS8LDyM7QfqJJC4JJ8nE0e7iANgmVZCrByZi/QEX8bbbPp4j+T7lynmHKRPhvEAUsd3zPn1BpHJZNGYLfoK6pMEbj/ZfF/YTjkUPrSzzRFdmwT2SvFnTCIsWf64NhXo8lQ5CioNHjRAWaepXPnJ2UexnjW0cfxDkc/Un5xX0faOykuO76LRmU2I7OBHtwrqhaJQG+nYTyN00bJ86zTrPIz8xjsoNtPaYQhUGpPbfUI9hlnmiT2FdQY6b66qxeeb98NA/ZnYPJEp2MmBLljMXKaA/668x/DiYned0iW5vvc0OYBIuGM6yIqLeE3e/0HWqjMuNAanWKR03C5Ym/+hUJvsDiqOWUULsBvJOj5Q6OqweQoSA+vY1VLdY/vBYVIt3FdQbwZPwkYx/Zs08uF72mAChheEahnu2P4qVF25BskVt67xnaeyft+gbZkd7EzoU6QymZ56RkPVAQA0c8/N6tHMBEPsU/f305k3Jni3w7u0vS3bIH1c381A3oMQgLs67jw/Ya3h5YhaphGPW8HaFQks0ybhBEK5lXopsa4FJ3bKL8oGsCV5XhJSO7j7JqZcI+6W7fuOw==
X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:;
 IPV:CAL; SFV:NSPM; H:xir-pvapexch01.xlnx.xilinx.com;
 PTR:unknown-80-198.xilinx.com; CAT:NONE;
 SFS:(13230022)(4636009)(346002)(136003)(396003)(376002)(39860400002)(451199015)(40470700004)(46966006)(36840700001)(44832011)(31686004)(5660300002)(40460700003)(2906002)(36756003)(53546011)(9786002)(186003)(8936002)(26005)(82740400003)(47076005)(36860700001)(41300700001)(40480700001)(2616005)(70206006)(8676002)(7636003)(70586007)(478600001)(356005)(4326008)(336012)(316002)(110136005)(54906003)(426003)(31696002)(82310400005)(83380400001)(50156003)(43740500002);
 DIR:OUT; SFP:1101; 
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 17:50:11.3140 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a9af0369-d9a0-4e9d-db03-08da9b3090c0
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198];
 Helo=[xir-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource: SN1NAM02FT0009.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB7432
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

On 9/20/2022 6:48 PM, Ferruh Yigit wrote:
> On 9/20/2022 11:42 AM, Robin Jarry wrote:
> 
>>
>> dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
>> merely prints multiple independent JSON lines which cannot be fed
>> directly to any JSON parser. Moreover, the script complexity is rather
>> high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
>> sections. Rewrite it so that it can produce valid JSON.
>>
>> Remove the PCI database parsing for PCI-ID to Vendor-Device names
>> conversion. This should be done by external scripts (if really needed).
>>
>> Here are some examples of use with jq:
>>
>> Get the complete info for a given driver:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.name == "dmadev_idxd_pci")'
>>   {
>>     "name": "dmadev_idxd_pci",
>>     "params": "max_queues=0",
>>     "kmod": "vfio-pci",
>>     "pci_ids": [
>>       {
>>         "vendor": "8086",
>>         "device": "0b25"
>>       }
>>     ]
>>   }
>>
>> Get only the required kernel modules for a given driver:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.name == "net_i40e").kmod'
>>   "* igb_uio | uio_pci_generic | vfio-pci"
>>
>> Get only the required kernel modules for a given device:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == 
>> "1013").kmod'
>>   "* ib_uverbs & mlx5_core & mlx5_ib"
>>
>> Print the list of drivers which define multiple parameters without
>> space separators:
>>
>>   ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \
>>     jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | 
>> {name, params}'
>>   ...
>>
>> The script passes flake8, black, isort and pylint checks.
>>
>> I have tested this with a matrix of python/pyelftools versions:
>>
>>                               pyelftools
>>                 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
>>           3.6     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.7     ok   ok   ok   ok   ok   ok   ok   ok
>>    Python 3.8     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.9     ok   ok   ok   ok   ok   ok   ok   ok
>>           3.10  fail fail fail fail   ok   ok   ok   ok
>>
>> All failures with python 3.10 are related to the same issue:
>>
>>    File "elftools/construct/lib/container.py", line 5, in <module>
>>      from collections import MutableMapping
>>    ImportError: cannot import name 'MutableMapping' from 'collections'
>>
>> Python 3.10 support is only available since pyelftools 0.26. The script
>> will only work with Python 3.6 and later. Update the minimal system
>> requirements and release notes.
>>
>> NB: The output produced by the legacy -r/--raw flag can be obtained with
>> the following command:
>>
>>    strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p'
>>
>> Cc: Olivier Matz <olivier.matz@6wind.com>
>> Cc: Ferruh Yigit <ferruh.yigit@xilinx.com>
>> Cc: Bruce Richardson <bruce.richardson@intel.com>
>> Signed-off-by: Robin Jarry <rjarry@redhat.com>
> 
> <...>
> 
>> diff --git a/doc/guides/rel_notes/release_22_11.rst 
>> b/doc/guides/rel_notes/release_22_11.rst
>> index 8c021cf0505e..67054f5acdc9 100644
>> --- a/doc/guides/rel_notes/release_22_11.rst
>> +++ b/doc/guides/rel_notes/release_22_11.rst
>> @@ -84,6 +84,11 @@ API Changes
>>      Also, make sure to start the actual text at the margin.
>>      =======================================================
>>
>> +* The ``dpdk-pmdinfo.py`` script was rewritten to produce valid JSON 
>> only.
>> +  PCI-IDs parsing has been removed.
>> +  To get a similar output to the (now removed) ``-r/--raw`` flag, you 
>> may use the following command::
>> +
>> +     strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
>>
> 
> Empty line is missing (in case there will be a new version for some 
> other reason).
> 
> 
> Thanks for the update,
> Tested-by: Ferruh Yigit <ferruh.yigit@xilinx.com>
> 

Thomas, John,

Should we have documentation for usertools, since they are user facing, 
what do you think?
Can it be possible to find resource for it?

Thanks,
ferruh