From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0071.outbound.protection.outlook.com [104.47.42.71]) by dpdk.org (Postfix) with ESMTP id 321D8378B for ; Wed, 24 May 2017 03:02:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=studentuml.onmicrosoft.com; s=selector1-student-uml-edu; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=eB9kLymYADEmYWBojQXtMxhACYTMg/nHRArn4IPwYvM=; b=qDZWlw2MHkYC6lqEcp593nKy0b2pS/V8HC51O5ETuK3W97+tVsd3/7zj5rXqYY3VF/2Y5rCjySvApVzzblrh6/IGsfKMrzDZTahXZ8hW8GU9tFNSLC79KjI/ux+5WaCV7mvC9AasG7ZThyqj5tDSNIfqDErkOmnBtMfkFQjGSng= Received: from BN6PR02MB2706.namprd02.prod.outlook.com (10.175.95.20) by BN6PR02MB2708.namprd02.prod.outlook.com (10.175.95.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1101.14; Wed, 24 May 2017 01:01:59 +0000 Received: from BN6PR02MB2706.namprd02.prod.outlook.com ([10.175.95.20]) by BN6PR02MB2706.namprd02.prod.outlook.com ([10.175.95.20]) with mapi id 15.01.1101.021; Wed, 24 May 2017 01:01:59 +0000 From: "Wu, Xiaoban" To: Adrien Mazarguil CC: "users@dpdk.org" Thread-Topic: [dpdk-users] Mellanox 100G NIC, VF SR-IOV, docker container, EAL could not detect the device Thread-Index: AQHS04bl+l8wHsB7oEadpj/oNHqf76IBkYQAgAEYX9o= Date: Wed, 24 May 2017 01:01:58 +0000 Message-ID: References: , <20170523081214.GA1758@6wind.com> In-Reply-To: <20170523081214.GA1758@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=student.uml.edu; x-originating-ip: [2603:10b6:404:f5:cafe::16] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; BN6PR02MB2708; 7:2VPUTf4hFh/sZf8U8QG7ZZNJXiBS1G5ZqZBR0p/nLYkRNaULwl7kVGvQH7lKf5QOTj1GnjJfFy5oPchuTOsl01gRPqTh89HWPK8TwlR9wkk1VQA2mMB15bJohNt08tDlMhvCi+Q8GZDqKUrXAkizDtCtbqvA+sQAm8QYh8CVD/0RhymxaFoAMMCbjL3Zv9MapWAQVOlw4ggIiBHz8hUcvOdYwYqxiwbl7au3S8Q/BVOVdnN7I98wPF6hxAOE8ivji6fr7+4aCvAIGJGHBje0Lws9p35STF2lMxtKmrESAommJ56hTOpCiZCzLTWDTOzIRNf0PwbR1UXad+RficcVxg== x-ms-traffictypediagnostic: BN6PR02MB2708: x-ms-office365-filtering-correlation-id: 4384efe0-49db-4c9d-f48e-08d4a2407b02 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(201703131423075)(201703031133081); SRVR:BN6PR02MB2708; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(93001095)(6041248)(20161123562025)(20161123558100)(20161123555025)(20161123560025)(20161123564025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(6072148); SRVR:BN6PR02MB2708; BCL:0; PCL:0; RULEID:; SRVR:BN6PR02MB2708; x-forefront-prvs: 031763BCAF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(39450400003)(39410400002)(39400400002)(39840400002)(24454002)(377454003)(42882006)(2950100002)(72206003)(6916009)(478600001)(74316002)(53936002)(33656002)(6436002)(6606003)(7696004)(75432002)(55016002)(76176999)(54356999)(50986999)(229853002)(2900100001)(110136004)(38730400002)(6246003)(54896002)(9686003)(99286003)(7736002)(5660300001)(81166006)(102836003)(6116002)(86362001)(8676002)(25786009)(3280700002)(4326008)(53546009)(122556002)(2906002)(8936002)(189998001)(77096006)(6506006)(88552002)(3660700001)(19627405001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR02MB2708; H:BN6PR02MB2706.namprd02.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: student.uml.edu X-MS-Exchange-CrossTenant-originalarrivaltime: 24 May 2017 01:01:58.7056 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4c25b8a6-17f7-46f9-83f0-54734ab81fb1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR02MB2708 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Mellanox 100G NIC, VF SR-IOV, docker container, EAL could not detect the device X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 May 2017 01:02:02 -0000 Hi Adrien, Thanks very much for your reply. However, after the -w option is used, the EAL still could not find the VF d= evices. $dpdk-app -c 0x01 --socket-mem=3D128,128 --file-prefix=3D"docker1" -w 0000:= 83:00.1 EAL: Detected 12 lcore(s) EAL: Probing VFIO support... EAL: VFIO support initialized PMD: bnxt_rte_pmd_init() called for (null) Error, rte_eth_dev_configure() returns negative! Best wishes, Xiaoban ________________________________ From: Adrien Mazarguil Sent: Tuesday, May 23, 2017 4:12 AM To: Wu, Xiaoban Cc: users@dpdk.org Subject: Re: [dpdk-users] Mellanox 100G NIC, VF SR-IOV, docker container, E= AL could not detect the device On Tue, May 23, 2017 at 05:39:36AM +0000, Wu, Xiaoban wrote: > Dear DPDK users, > > > I am trying to use the VF of the Mellanox 100G NIC enabled by SR-IOV. The= following is what I have done. > > > 1. Add "intel_iommu=3Don iommu=3Dpt" to kernel command line option, = update-grub, and reboot > > 2. Install the MLNX-OFED, reboot > > 3. In default the card is in infiband mode, so I switched it to ethe= rnet mode (in order to run DPDK application) and reboot. > > 4. mst start > > mlxconfig -d /dev/mst/mt4115_pciconf0 q #query > mlxconfig -d /dev/mst/mt4115_pciconf0 set SRIOV_EN=3D1 NUM_OF_VF= S=3D1 > reboot > 5. echo 1 > /sys/bus/pci/devices/0000\:83\:00.0/mlx5_num_vfs > 6. modprobe vfio-pci > 7. dpdk-devbind.py --status > 8. dpdk-devbind.py -b vfio-pci 0000:83:00.1 > 9. ls -al /dev/vfio > 10. docker run -it --privileged --device=3D/dev/vfio/54:/dev/vfio/54= --device=3D/dev/vfio/vfio:/dev/vfio/vfio -v /mnt/huge/:/dev/hugepages/ -v = /var/run:/var/run $IMAGEID bash > 11. $dpdkapp -c 0x01 --socket-mem=3D128,128 --file-prefix=3D"docker1= " > > However, in the EAL part, it does not list any usable devices > EAL: Detected 12 lcore(s) > EAL: Probing VFIO support... > EAL: VFIO support initialized > PMD: bnxt_rte_pmd_init() called for (null) > Error, rte_eth_dev_configure() returns negative! > > Can anybody please point out any possible solution? Looking forward to yo= ur reply. Thanks very much for your help. Seems like the issue is not related to your mlx5 device. From the above log it appears that you also have a bnxt device on that system which DPDK detects and attempts to use as it is running in blacklist mode. Perhaps tha= t device was not configured properly. Try to white-list the devices you want to use by explicitly providing their PCI bus addresses through -w arguments instead. -- Adrien Mazarguil 6WIND