From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E40A1A052B for ; Tue, 28 Jul 2020 18:23:49 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 86B342B8D; Tue, 28 Jul 2020 18:23:49 +0200 (CEST) Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by dpdk.org (Postfix) with ESMTP id E09F62B8D for ; Tue, 28 Jul 2020 18:23:47 +0200 (CEST) Received: by mail-pl1-f172.google.com with SMTP id q17so10110874pls.9 for ; Tue, 28 Jul 2020 09:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3eVC84xwVkhsE01JDuvRI1j/Yxx3FI2gmTNFu2fEBrs=; b=jfU3TbWZgZdwaNT6HDCsWWzMKXBAi8zl2vmb20FDUaqSZQcQiEjriYIj3vGZBC9m0x F91vub/UMXcWyVMnQd2dkKLVjgISBtB2ID+R2F0go8L7HEmlQ+FzPQieUF3EUXKXDnqd tAbqAjbcLxZ/WO0tAQlCvQImZ8bHQR8bYUFA1MSmOsgsTkJSCR8/Wz9U40V0kPaVFAjS x/MjfNFLwqaNOpzXFLfFLX2p2QJwDPX43wz1gDomanMbP1VorszEoABdn5F02aqvShpa sqcGL65jLOQ2LD4dY+qq/ed8fT1aqtoafvw/qcB6hRj7HoFFxag6szNhzQAmrMTyQMNo b4pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3eVC84xwVkhsE01JDuvRI1j/Yxx3FI2gmTNFu2fEBrs=; b=SlInUhQczaQFJtGiqrPPrzSqzilm7SRfUvegAx8sKZ9q+zcZ4hH/r5aSHotjxiYoM8 78PqnScf/qNJ91soRzuHEgpyF/4rcNTFfitnokzaTYy/X2zNtL/m4PmuPvoXTKKhbZS3 RgYIEGFHZGL60wXO10UGzGQ7beIeHq+jl/pmZn2/e8kJqG4IETspnN/ig64UkxUsybeg 0XNooLF2ShXK6CQu8/lcrXiE45tgjb8tySxlW5nXvl5vuwF+4fVPEARPTcUj8HP2FQLf fHE0sGCVRwtHoTH8P0PcYihPqw6Uxa2VbUHChaEHymfEYQaCCcbOXG2PksrEgA/K+EcF Lctw== X-Gm-Message-State: AOAM531CY72jTECALeSgd1n2ELwVLbJuNfMNNMl5rDSWejE/6z2btxri LgyP3CO7EP7nR4lYEpdUSl9Jng== X-Google-Smtp-Source: ABdhPJyB7bzsOowHPEn6wpZQWHt+k5YiUIzY+l62JuRyZKAjuQMzc5PKPfUSDPHDYMHb/dKpGk5C4Q== X-Received: by 2002:a17:902:e993:: with SMTP id f19mr24164994plb.305.1595953426814; Tue, 28 Jul 2020 09:23:46 -0700 (PDT) Received: from hermes.lan (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id b13sm8817236pgd.36.2020.07.28.09.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jul 2020 09:23:46 -0700 (PDT) Date: Tue, 28 Jul 2020 09:23:37 -0700 From: Stephen Hemminger To: Dikshant Chitkara Cc: "Varghese, Vipin" , "users@dpdk.org" , "dev@dpdk.org" , "Amir Ilan" , Veeresh Patil Message-ID: <20200728092337.076df2e3@hermes.lan> In-Reply-To: <7c8c706747384c869ff055c6af0c6fa3@Airspan.com> References: <20200714092902.31e95f83@hermes.lan> <7c8c706747384c869ff055c6af0c6fa3@Airspan.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] [dpdk-dev] DPDK PDUMP Issue X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" On Tue, 28 Jul 2020 14:51:58 +0000 Dikshant Chitkara wrote: > Hi Team, > > With fix suggest in prev mail thread at testpmd side, PDUMP works with testpmd. > > However when we try to run our own primary app with PDUMP as a secondary process, PDUMP console comes up, however it does not capture any packets. > > Changes made at primary app side: > 1. PDUMP initialised just after rte eal init: > > ret_pdump = rte_pdump_init(); > if (ret_pdump < 0) { > printf("rte_pdump_init failed\n"); > } > else > { > printf("rte_pdump_init success\n"); > } > > 2. Makefile modified to add all relevant files to be linked: > > LIB_SO =-L$(DPDK_LIB) -lrte_mbuf -lrte_eal -lnuma -lrte_pmd_pcap -lrte_pdump -lrte_pmd_i40e -lrte_eal -lrte_ring -lrte_mempool -lrte_cryptodev -lrte_ethdev -lrte_mbuf -lrte_mempool_ring -lrte_member -lrte_eventdev -lrte_bus_vmbus -lrte_pci -lrte_bus_pci -lrte_bus_vdev -lrte_net -lrte_distributor -lrte_reorder -lrte_kni -lrte_pipeline -lrte_table -lrte_timer -lrte_hash -lrte_jobstats -lrte_lpm -lrte_power -lrte_acl -lrte_meter -lrte_sch > > Even tried to link final DPDK lib so as well: > LIB_SO =-L$(DPDK_LIB) -Wl,--whole-archive -ldpdk -Wl,--no-whole-archive -L/usr/lib/x86_64-linux-gnu/ -fPIC > > It seems both apps are able to connect over mp socket but pdump unable to capture data sent from primary app. > > Can you tell us what else we might be missing here. > > Note: We are using DPDK 19_05 built in shared lib mode. > > > Primary app console: > > EAL: Probing VFIO support... > EAL: PCI device 0000:41:00.0 on NUMA socket 0 > EAL: probe driver: 8086:37d2 net_i40e > EAL: PCI device 0000:41:00.1 on NUMA socket 0 > EAL: probe driver: 8086:37d2 net_i40e > EAL: PCI device 0000:86:00.0 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.1 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.2 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.3 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:88:00.0 on NUMA socket 1 > EAL: probe driver: 8086:158b net_i40e > EAL: PCI device 0000:88:00.1 on NUMA socket 1 > EAL: probe driver: 8086:158b net_i40e > rte_pdump_init success > pool_create successed > DPDK TX Ethernet device (destination) opened successfully > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > Before send burst > Packets sent. Count=32 > >>> > > Pdump console: > > [root@flexran3 x86_64-native-linux-icc]# ./app/dpdk-pdump -d librte_pmd_i40e.so -d librte_pmd_pcap.so -- --pdump 'port=0,queue=*,tx-dev=/home/dchitkara/capture.pcap' > EAL: Detected 80 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_45398_5ed780258ea46 > EAL: Probing VFIO support... > EAL: PCI device 0000:41:00.0 on NUMA socket 0 > EAL: probe driver: 8086:37d2 net_i40e > EAL: PCI device 0000:41:00.1 on NUMA socket 0 > EAL: probe driver: 8086:37d2 net_i40e > EAL: PCI device 0000:86:00.0 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.1 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.2 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:86:00.3 on NUMA socket 1 > EAL: probe driver: 8086:1572 net_i40e > EAL: PCI device 0000:88:00.0 on NUMA socket 1 > EAL: probe driver: 8086:158b net_i40e > EAL: PCI device 0000:88:00.1 on NUMA socket 1 > EAL: probe driver: 8086:158b net_i40e > Port 1 MAC: 02 70 63 61 70 00 > core (0), capture for (1) tuples > - port 0 device ((null)) queue 65535 > ^C > > Signal 2 received, preparing to exit... > ##### PDUMP DEBUG STATS ##### > -packets dequeued: 0 > -packets transmitted to vdev: 0 > -packets freed: 0 > [root@flexran3 x86_64-native-linux-icc]# > > > > > -----Original Message----- > From: Varghese, Vipin > Sent: 22 July 2020 09:38 > To: Stephen Hemminger ; Dikshant Chitkara > Cc: users@dpdk.org; dev@dpdk.org; Amir Ilan ; Veeresh Patil > Subject: RE: [dpdk-dev] DPDK PDUMP Issue > > NOT FROM AIRSPAN - Caution - External from: vipin.varghese@intel.com Snipped > > > Dikshant Chitkara wrote: > > > > > Hi Team, > > > > > > We are facing an issue w.r.t to using DPDK PDUMP utility and want to > > > check if > > you can help us in this regard. > > > > > > This issue is specific to DPDK 19.05 version or may be newer > > > versions and > > was not found on DPDK 18.08 version. > > > > > > The issue is that when we built DPDK in shared library mode and try > > > to run > > PDUMP we get following error on PDUMP console: > > > > > > EAL: Failed to hotplug add device > > > EAL: Error - exiting with code: 1 > > > Cause: vdev creation failed > > > > > > On Debugging above error message, I found that inside > > > rte_dev_probe() > > function in the file lib/librte_eal/common/eal_common_dev.c , > > req.result = - > > 14 which is non-zero value and points to enum EFAULT (Bad address) > > value, thus leading to above error. > > > > > > However, when I ran PDUMP in DPDK 18.08, I did not observe any such > > > issue > > and it worked fine. > > > > > > DPDK was built with following flags set in config/common_base file: > > > > > > CONFIG_RTE_LIBRTE_PMD_PCAP=y > > > CONFIG_RTE_LIBRTE_PDUMP=y > > > > > > I ran PDUMP with testpmd as primary application and was trying to > > > send > > some traffic on the Ethernet dpdk port using testpmd and capture it > > using pdump. > > > > > > Attached are the logs for both testpmd and pdump in both DPDK19_05 > > > and > > DPDK_18_08 version for your reference. > > > > > > Let us know if you can help in this regard. > > > > > > Thanks, > > > Dikshant > > > > > > > Pdump runs as secondary process, what is your primary process. > > Is the secondary process using the same exact versions of DPDK and > > libraries as the primary process? > > The secondary process needs access to the same devices and drivers as > > the primary process to work. > > Based on the log files shared, I have made assumptions as > > 1. Primary: testpmd > 2. DPDK version: 19.05 > 3. SHARED LIB in .config: yes > 4. export LD_LIBRARY_PATH: yes > > Using arguments passed in primary, I faced error ``` > EAL: Error - exiting with code: 1 > Cause: Creation of mbuf pool for socket 0 failed: Invalid argument > > Cause: Mempool creation failed: Invalid argument ``` > > Hence had to modify with ` -d librte_mempool_ring.so`. Using arguments passed in secondary, I faced similar error too. > > With these corrections, I am able to get similar error shared as ``` > EAL: failed to parse device "vdev:net_pcap_tx_0" > EAL: Failed to hotplug add device on primary > EAL: Error - exiting with code: 1 > Cause: vdev creation failed > ``` > > Checking the testpmd Makefile, it is narrowed down to missing PCAP_PMD build factor. I rebuild testpmd with ` LD_FLAGS="-lrte_pmd_pcap" make` and re-ran the same. With this I can get it working properly. > > Ideally there should had been an entry in Makefile as ``` ifeq ($(CONFIG_RTE_LIBRTE_PMD_PCAP),y) LDLIBS += -lrte_pmd_pcap endif ``` > > reference: This is similar to the error and solution provided in ` https://stackoverflow.com/questions/62795017/dpdk-pdump-failed-to-hotplug-add-device/62869287#62869287` > > The primary/secondary work is that both processes have to discover the same memory areas and devices. The NUMA thing might be related, does your system have multiple cpu sockets. What does the cpu_layout python script show.