From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 415FEA0524; Mon, 7 Dec 2020 11:48:44 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9FCE2F12; Mon, 7 Dec 2020 11:48:41 +0100 (CET) Received: from mailgw02.pantheon.sk (mailgw01.pantheon.sk [46.229.239.26]) by dpdk.org (Postfix) with ESMTP id 98C1FCF3 for ; Mon, 7 Dec 2020 11:48:40 +0100 (CET) Received: from mailgw02.pantheon.sk (localhost.localdomain [127.0.0.1]) by mailgw02.pantheon.sk (Proxmox) with ESMTP id 0C01A184050; Mon, 7 Dec 2020 11:48:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; h=cc:cc:content-type:content-type:date:from:from:message-id :mime-version:reply-to:subject:subject:to:to; s=dkim; bh=/6/K50z yL9dMDE14o8JJGJ/4RIRO92RT5LZQ3kYNKC4=; b=q803t8TPgtpiXztVbQuJfHu o5kqbNA/TxDempFxOik7WjtXnjOKyCuZRp4IHgkQxy2HDrpc+x1CFwjEuA4edJNv K1/EdzvIDPNimUu20c7FoU212Z55olfMSe/tZUOuzPWuo+FNSLfOC5G2/8aVsOla ZJLaVYaMftBt08bmJbJ7AMKM/RL5eRMgJSW8AllcRRaXOZpQX8Yuf21QJSknsx9X 1X3L90egwnmXMNTwMebrCs/m1/aLvD0zRLvfsAhjJJJ9JXio41IpWPqQJA5+Nvla mdk29sJl1lATqlDJpE509qi1Q55o6UZWIYuyO9KuFBaBya+CEi30U7Oi7eP26Lg= = From: =?iso-8859-2?Q?Juraj_Linke=B9?= To: "dev@dpdk.org" CC: "Kinsella, Ray" , "Andrew Yourtchenko (ayourtch)" Thread-Topic: Faulty VF initialization during DPDK startup when multiple DPDK instances use different VFs with the same PF Thread-Index: AdbMhM+q2mgYN4QqTnylVPeB7DKS/Q== Date: Mon, 7 Dec 2020 10:48:38 +0000 Message-ID: <9d5b0d3a3bb648d5a296eb794006db14@pantheon.tech> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.101.4.10] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Faulty VF initialization during DPDK startup when multiple DPDK instances use different VFs with the same PF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi DPDK devs, A while back I've submitted this bug: https://bugs.dpdk.org/show_bug.cgi?id= =3D578 and now we have a pretty good idea where the issue stems from. TL;DL= : it seems to be in either XL710 firmware or i40e driver, with downstream e= ffects which we may need to address in DPDK. What is the issue? We're using an XL710 NIC with SR-IOV setup with multiple virtual functions = (VFs) that belong to the same physical function (PF). We're observing inter= mittent failures when multiple DPDK EAL instances are trying to initialize = different VFs from the PF. One of the failures looks like this: i40evf_check_api_version(): PF/VF API version mismatch:(0.0)-(1.1) This results in VPP (which uses DPDK to initialize these VFs) not being abl= e to use the VFs. There an associated syslog: [Thu Dec 3 02:30:56 2020] i40e 0000:05:00.1: Unable to send the message to= VF 49 aq_err 12 Digging in the sources we've found that this is the error message: https://elixir.bootlin.com/linux/v4.15/source/drivers/net/ethernet/intel/i4= 0evf/i40e_adminq_cmd.h#L115 This suggests it's an issue with either the driver or firmware and that lea= ds us to two questions: 1) Is this an expected condition to happen? What is the reason for this con= tention and is it normal to have it, and what is the expected correct behav= ior of the calling code? 2) If "yes" to the previous question - then, since the caller in this case = initialization code of DPDK, should we address it there (e.g. some retries = or a lock)? Are there any Intel (or SR-IOV) experts who could help with answering the f= irst question? Or is it possible that no matter what the expected behavior = is should we address it in DPDK? This is just a short description, there's more information in Bugzilla. Thanks, Juraj