From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 080AAA00C3; Tue, 20 Sep 2022 12:10:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5D6A40DFB; Tue, 20 Sep 2022 12:10:45 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 5E7D04069B for ; Tue, 20 Sep 2022 12:10:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RNYdp8eMjEv4f0F8tLK61OmeNQ0vV+um4JH17NtIUXO5HK3He+ISgNDWdKoTG3VMsOX8Q13+j71y99g+4n8cy9d7EbUbOJu8Xu3GTj6KWF4GlZ7MtHHV5akO5/CUgN4jbp3Be9sXLOEJUcE8eFGBRfI4atEiFVDQ0EBhFczp/B+vvgT9HF68vNGmVhz5yJV7lzcXwRKU8rPoAVLEr0W1KB9xAAcYa4k5lfyoQlIULjUv4xKBtYeqLlUrDA0YM6nlhk0mMlmUqT5JsPjyCj3Beq6/uBhBx5AhocKk+kuzXaIj3Y5EIRq4ldWU/FaZayX95tsBVB88PJP112JN69cNPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OpI9Q4kILrYb+tj0At9TaJ1lKmVVfTv76KusZ4lCBPE=; b=kTDnUpOqzACge6wbIyL+FKD04vmIGjQmXkh98W+ozfWqNoI9OHs4Jzo9IMFSH2l0iqyX3oyPoyMHFRuxadJHB6tPFJpUk44t0kWieJafehaFwx4HlP2XakEaa+AWkrlUhD0ffxMxzkLoJGM7y1o7Zj96OkxgKSPSW5bEHh7uwqa6K52aP3WMI+CEQBZ6KszK7CrvWkM2uyHbcKEXsfGXSy+gPx99N0OjWyqWyhpsnxhGUjaflVlMM6ham/E3ilcQ38huA1bE2QRcS0T35uwXoY6R77tDS9w9ID0O7Gzz5dJpONM2Sb1t7h6dsMAdV8eIRmji3M0d6yE3N7d+EReU4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=redhat.com smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OpI9Q4kILrYb+tj0At9TaJ1lKmVVfTv76KusZ4lCBPE=; b=SeM/sSVBSe8E3zOoLEB0POxvVCxAlDes5k1Z3zLFkJtfH2Ik5jOf6ElfMwWIT0q+/puuFdtuzYXq9uzDRXInrA3vs37pAeYWKLXpKX1JnNFYv8ltVxm44gqnzA2yRq0cBEu9jn6eSZSuKFNPjeCjYMUYim0Rjh39KPdPVQ0Zd+Y= Received: from BN0PR04CA0133.namprd04.prod.outlook.com (2603:10b6:408:ed::18) by DM6PR02MB6844.namprd02.prod.outlook.com (2603:10b6:5:21a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.21; Tue, 20 Sep 2022 10:10:41 +0000 Received: from BN1NAM02FT005.eop-nam02.prod.protection.outlook.com (2603:10b6:408:ed:cafe::2c) by BN0PR04CA0133.outlook.office365.com (2603:10b6:408:ed::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.21 via Frontend Transport; Tue, 20 Sep 2022 10:10:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch01.xlnx.xilinx.com; pr=C Received: from xir-pvapexch01.xlnx.xilinx.com (149.199.80.198) by BN1NAM02FT005.mail.protection.outlook.com (10.13.2.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5632.12 via Frontend Transport; Tue, 20 Sep 2022 10:10:40 +0000 Received: from xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 20 Sep 2022 11:10:40 +0100 Received: from smtp.xilinx.com (172.21.105.197) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Tue, 20 Sep 2022 11:10:39 +0100 Envelope-to: rjarry@redhat.com, dev@dpdk.org, jerinj@marvell.com, olivier.matz@6wind.com, bruce.richardson@intel.com, rmody@marvell.com, dsinghrawat@marvell.com, asekhar@marvell.com, pbhagavatula@marvell.com, ndabilpuram@marvell.com, kirankumark@marvell.com, skori@marvell.com, skoteshwar@marvell.com Received: from [10.71.194.74] (port=60807) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oaaD9-0006Lm-0P; Tue, 20 Sep 2022 11:10:39 +0100 Message-ID: <27a4885e-1fa3-9856-ccb2-e8bb82a96797@xilinx.com> Date: Tue, 20 Sep 2022 11:10:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Subject: Re: [PATCH v3] usertools: rewrite pmdinfo Content-Language: en-US To: Robin Jarry , , Jerin Jacob Kollanukkaran CC: Olivier Matz , Bruce Richardson , Rasesh Mody , Devendra Singh Rawat , Ashwin Sekhar T K , Pavan Nikhilesh , Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao References: <20220913105811.84384-1-robin@jarry.cc> <20220920090818.470814-1-rjarry@redhat.com> From: Ferruh Yigit In-Reply-To: <20220920090818.470814-1-rjarry@redhat.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1NAM02FT005:EE_|DM6PR02MB6844:EE_ X-MS-Office365-Filtering-Correlation-Id: a3cd3600-7c78-4c5b-0ed1-08da9af05f7a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +m9pfNX/VUIjPypB3JLxqRgl+HbEPW6e1q4zKeE+J60yTwEWie4vt4NQ/kQei9KzYxnRivV3GkdBUBz4pHrj9MUe50pAIKQbGQuxYfe8e0Aii0XfpsfFxXIt/6TSzOJx8AJsrHqz6EH+eCSSHgNtefjY/aYOqH31xKW/SbImLB/M3l/feYg3/U2+P8RQG+z3IdLYyvNISW67excD/Hy1BO9B6/hxXaElVBhg2UlB9M5TZ20HRLey6HqkIZEaJV8BSPJPi3jq6J3wfFDlsC2UJq4N193+9PihP+T0WXyCXDimUi/GY4541QxQLMlKCJmI1/aU/whnVUsImkl7idDSuud0Y15GD0qCTyudcfsO8O98STJlNoyO9QTBmNBpbJrZsqtogQqfBrBB5OG8Nyj3LZMyz4zIcqsJPGL3c08GD2YLuXAcTRUMsfHN/ir622u41/8pE3FqGOcA1VT8Ppajnqr10DAAjZemrrCPqxQDKb0E3tmllzeMDbJjtw1DxKz4zbBy/DevKHMADWeN7eUDBI46/GKr+4oPTCbVXW9gSuVLxtIh4JZoOhG+3epxQCRUlrc5krSVmgaOKNZ5aRtYyacWchAgADu53y8iyxk0Zxdl67e7QnOXNZzAGPYf7McHM8abo6GLw5xth8+MPzcRSA7cEi5cWhpI2CQMokCLU8NGpw3BM0B1b9ttNC/ZHH+vmECBEL5faUHiQDVsmIDsEJvH42Kw4nPLwz9a0OBaB3p5y9ZscuH8sMsPu3Z8u5kDuXkf7FaWjyHjyBIB2oeeIG+hoL0jIkO+jeqpJL5/PAT2OefoABwXl8Sip7aGBMvKozRIub5iNlWDAJsuWwAUkQ== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch01.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(346002)(136003)(376002)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(5660300002)(336012)(40460700003)(186003)(2616005)(82310400005)(316002)(44832011)(7416002)(31686004)(54906003)(53546011)(356005)(9786002)(8936002)(26005)(40480700001)(47076005)(4326008)(36756003)(2906002)(31696002)(110136005)(82740400003)(478600001)(426003)(83380400001)(36860700001)(8676002)(70586007)(70206006)(41300700001)(7636003)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 10:10:40.9291 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a3cd3600-7c78-4c5b-0ed1-08da9af05f7a X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: BN1NAM02FT005.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB6844 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/20/2022 10:08 AM, Robin Jarry wrote: > CAUTION: This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. > > > dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag > merely prints multiple independent JSON lines which cannot be fed > directly to any JSON parser. Moreover, the script complexity is rather > high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF > sections. Rewrite it so that it can produce valid JSON. > > Remove the PCI database parsing for PCI-ID to Vendor-Device names > conversion. This should be done by external scripts (if really needed). > > Here are some examples of use with jq: > > Get the complete info for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "dmadev_idxd_pci")' > { > "name": "dmadev_idxd_pci", > "params": "max_queues=0", > "kmod": "vfio-pci", > "pci_ids": [ > { > "vendor": "8086", > "device": "0b25", > "subsystem_vendor": "ffff", > "subsystem_device": "ffff" > } > ] > } > > Get only the required kernel modules for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "net_i40e").kmod' > "* igb_uio | uio_pci_generic | vfio-pci" > > Get only the required kernel modules for a given device: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == "1013").kmod' > "* ib_uverbs & mlx5_core & mlx5_ib" > > Print the list of drivers which define multiple parameters without > string separators: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}' > ... > > The script passes flake8, black, isort and pylint checks. > > I have tested this with a matrix of python/pyelftools versions: > > pyelftools > 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 > 3.6 ok ok ok ok ok ok ok ok > 3.7 ok ok ok ok ok ok ok ok > Python 3.8 ok ok ok ok ok ok ok ok > 3.9 ok ok ok ok ok ok ok ok > 3.10 fail fail fail fail ok ok ok ok > > All failures with python 3.10 are related to the same issue: > > File "elftools/construct/lib/container.py", line 5, in > from collections import MutableMapping > ImportError: cannot import name 'MutableMapping' from 'collections' > > Python 3.10 support is only available since pyelftools 0.26. The script > will only work with Python 3.6 and later. Update the minimal system > requirements and release notes. > > NB: The output produced by the legacy -r/--raw flag can be obtained with > the following command: > > strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p' > > Cc: Olivier Matz > Cc: Ferruh Yigit > Cc: Bruce Richardson > Signed-off-by: Robin Jarry For 'subsystem_vendor' & 'subsystem_device', the value "ffff" means it is not explicitly defined, so it gets default value. What do you think to omit those as well, when value is "ffff", to reduce noise on the output? BTW, I have detected some duplicates in the output, like [1], [2] & [3]. It seems related to the duplicates in the code, cc'ed maintainers. [1]: { "name": "net_qede", "kmod": "* igb_uio | uio_pci_generic | vfio-pci", "pci_ids": [ { "vendor": "1077", "device": "1634", "subsystem_vendor": "ffff", "subsystem_device": "ffff" }, { "vendor": "1077", "device": "1629", "subsystem_vendor": "ffff", "subsystem_device": "ffff" }, { "vendor": "1077", "device": "1634", "subsystem_vendor": "ffff", "subsystem_device": "ffff" }, ... [2] { "name": "mempool_cnxk", "params": "max_pools=<128-1048576>", "kmod": "vfio-pci", "pci_ids": [ { "vendor": "177d", "device": "a0fb", "subsystem_vendor": "ffff", "subsystem_device": "b900" }, { "vendor": "177d", "device": "a0fb", "subsystem_vendor": "ffff", "subsystem_device": "b900" }, ... [3] { "name": "net_cn10k", "kmod": "vfio-pci", "pci_ids": [ { "vendor": "177d", "device": "a063", "subsystem_vendor": "ffff", "subsystem_device": "b900" }, { "vendor": "177d", "device": "a063", "subsystem_vendor": "ffff", "subsystem_device": "b900" }, ...