From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 545DEA0C45; Mon, 6 Dec 2021 02:52:38 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CEDEC4013F; Mon, 6 Dec 2021 02:52:37 +0100 (CET) Received: from smtp-gw.pt.net (smtp-gw.pt.net [206.210.194.15]) by mails.dpdk.org (Postfix) with ESMTP id 1767940040 for ; Mon, 6 Dec 2021 02:52:35 +0100 (CET) X-ASG-Debug-ID: 1638755553-09411a39e1a42fb0001-TfluYd Received: from mail.pt.net (mail.pt.net [206.210.194.11]) by smtp-gw.pt.net with ESMTP id NoCt8nqoz8YE0bs3 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 05 Dec 2021 19:52:34 -0600 (CST) X-Barracuda-Envelope-From: lew@donzis.com X-Barracuda-Effective-Source-IP: mail.pt.net[206.210.194.11] X-Barracuda-Apparent-Source-IP: 206.210.194.11 Received: from localhost (localhost [IPv6:::1]) by mail.pt.net (Postfix) with ESMTP id E1D64D33485; Sun, 5 Dec 2021 19:52:33 -0600 (CST) Received: from mail.pt.net ([IPv6:::1]) by localhost (mail.pt.net [IPv6:::1]) (amavisd-new, port 10032) with ESMTP id V3dLNLcqu1uR; Sun, 5 Dec 2021 19:52:33 -0600 (CST) Received: from localhost (localhost [IPv6:::1]) by mail.pt.net (Postfix) with ESMTP id 8C05AD332EF; Sun, 5 Dec 2021 19:52:33 -0600 (CST) X-Virus-Scanned: amavisd-new at pt.net Received: from mail.pt.net ([IPv6:::1]) by localhost (mail.pt.net [IPv6:::1]) (amavisd-new, port 10026) with ESMTP id KPoRwF6hnnKZ; Sun, 5 Dec 2021 19:52:33 -0600 (CST) Received: from mail.pt.net (mail.pt.net [206.210.194.11]) by mail.pt.net (Postfix) with ESMTP id 7075DD33483; Sun, 5 Dec 2021 19:52:33 -0600 (CST) Date: Sun, 5 Dec 2021 19:52:33 -0600 (CST) From: Lewis Donzis To: Bruce Richardson Cc: dev , yongwang@vmware.com Message-ID: <1909271468.2730688.1638755553311.JavaMail.zimbra@donzis.com> In-Reply-To: References: <2134779104.413217.1638218715124.JavaMail.zimbra@donzis.com> Subject: Re: vmxnet3 no longer functional on DPDK 21.11 MIME-Version: 1.0 X-ASG-Orig-Subj: Re: vmxnet3 no longer functional on DPDK 21.11 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Originating-IP: [206.210.194.11] X-Mailer: Zimbra 8.8.15_GA_4173 (ZimbraWebClient - GC96 (Mac)/8.8.15_GA_4177) Thread-Topic: vmxnet3 no longer functional on DPDK 21.11 Thread-Index: W1U5Zk3/a4w+K56u0Pw98Bdt297i5w== X-Barracuda-Connect: mail.pt.net[206.210.194.11] X-Barracuda-Start-Time: 1638755553 X-Barracuda-Encrypted: TLS_AES_256_GCM_SHA384 X-Barracuda-URL: https://smtp-gw.pt.net:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at pt.net X-Barracuda-Scan-Msg-Size: 5960 X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.50 X-Barracuda-Spam-Status: No, SCORE=0.50 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.94435 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org ----- On Nov 30, 2021, at 7:42 AM, Bruce Richardson bruce.richardson@intel.com wrote: > On Mon, Nov 29, 2021 at 02:45:15PM -0600, Lewis Donzis wrote: >> Hello. >> We just upgraded from 21.08 to 21.11 and it's rather astounding the >> number of incompatible changes in three months. Not a big deal, just >> kind of a surprise, that's all. >> Anyway, the problem is that the vmxnet3 driver is no longer functional >> on FreeBSD. >> In drivers/net/vmxnet3/vmxnet3_ethdev.c, vmxnet3_dev_start() gets an >> error calling rte_intr_enable(). So it logs "interrupt enable failed" >> and returns an error. >> In lib/eal/freebsd/eal_interrupts.c, rte_intr_enable() is returning an >> error because rte_intr_dev_fd_get(intr_handle) is returning -1. >> I don't see how that could ever return anything other than -1 since it >> appears that there is no code that ever calls rte_intr_dev_fd_set() >> with a value other than -1 on FreeBSD. Also weird to me is that even >> if it didn't get an error, the switch statement that follows looks like >> it will return an error in every case. >> Nonetheless, it worked in 21.08, and I can't quite see why the >> difference, so I must be missing something. >> For the moment, I just commented the "return -EIO" in vmxnet3_ethdev.c, >> and it's now working again, but that's obviously not the correct >> solution. >> Can someone who's knowledgable about this mechanism perhaps explain a >> little bit about what's going on? I'll be happy to help troubleshoot. >> It seems like it must be something simple, but I just don't see it yet. > > Hi > > if you have the chance, it would be useful if you could use "git bisect" to > identify the commit in 21.11 that broke this driver. Looking through the > logs for 21.11 I can't identify any particular likely-looking commit, so > bisect is likely a good way to start looking into this. > > Regards, > /Bruce Hi, Bruce. git bisect is very time-consuming and very cool! I went back to 21.08, about 1100 commits, and worked through the process, but then I realized that I had forgotten to run ninja on one of the steps, so I did it again. I also re-checked it after the bisect, just to make sure that c87d435a4d79739c0cec2ed280b94b41cb908af7 is good, and 7a0935239b9eb817c65c03554a9954ddb8ea5044 is bad. Thanks, lew Here's the result: root@fbdev:/usr/local/share/dpdk-git # git bisect start root@fbdev:/usr/local/share/dpdk-git # git bisect bad root@fbdev:/usr/local/share/dpdk-git # git bisect good 74bd4072996e64b0051d24d8d641554d225db196 Bisecting: 556 revisions left to test after this (roughly 9 steps) [e2a289a788c0a128a15bc0f1099af7c031201ac5] net/ngbe: add mailbox process operations root@fbdev:/usr/local/share/dpdk-git # git bisect bad Bisecting: 277 revisions left to test after this (roughly 8 steps) [5906be5af6570db8b70b307c96aace0b096d1a2c] ethdev: fix ID spelling in comments and log messages root@fbdev:/usr/local/share/dpdk-git # git bisect bad Bisecting: 138 revisions left to test after this (roughly 7 steps) [a7c236b894a848c7bb9afb773a7e3c13615abaa8] net/cnxk: support meter ops get root@fbdev:/usr/local/share/dpdk-git # git bisect bad Bisecting: 69 revisions left to test after this (roughly 6 steps) [14fc81aed73842d976dd19a93ca47e22d61c1759] ethdev: update modify field flow action root@fbdev:/usr/local/share/dpdk-git # git bisect bad Bisecting: 34 revisions left to test after this (roughly 5 steps) [cdea571becb4dabf9962455f671af0c99594e380] common/sfc_efx/base: add flag to use Rx prefix user flag root@fbdev:/usr/local/share/dpdk-git # git bisect good Bisecting: 17 revisions left to test after this (roughly 4 steps) [7a0935239b9eb817c65c03554a9954ddb8ea5044] ethdev: make fast-path functions to use new flat array root@fbdev:/usr/local/share/dpdk-git # git bisect bad Bisecting: 8 revisions left to test after this (roughly 3 steps) [012bf708c20f4b23d055717e28f8de74887113d8] net/sfc: support group flows in tunnel offload root@fbdev:/usr/local/share/dpdk-git # git bisect good Bisecting: 4 revisions left to test after this (roughly 2 steps) [9df2d8f5cc9653d6413cb2240c067ea455ab7c3c] net/sfc: support counters in tunnel offload jump rules root@fbdev:/usr/local/share/dpdk-git # git bisect good Bisecting: 2 revisions left to test after this (roughly 1 step) [c024496ae8c8c075b0d0a3b43119475787b24b45] ethdev: allocate max space for internal queue array root@fbdev:/usr/local/share/dpdk-git # git bisect good Bisecting: 0 revisions left to test after this (roughly 1 step) [c87d435a4d79739c0cec2ed280b94b41cb908af7] ethdev: copy fast-path API into separate structure root@fbdev:/usr/local/share/dpdk-git # git bisect good 7a0935239b9eb817c65c03554a9954ddb8ea5044 is the first bad commit commit 7a0935239b9eb817c65c03554a9954ddb8ea5044 Author: Konstantin Ananyev Date: Wed Oct 13 14:37:02 2021 +0100 ethdev: make fast-path functions to use new flat array Rework fast-path ethdev functions to use rte_eth_fp_ops[]. While it is an API/ABI breakage, this change is intended to be transparent for both users (no changes in user app is required) and PMD developers (no changes in PMD is required). One extra thing to note - RX/TX callback invocation will cause extra function call with these changes. That might cause some insignificant slowdown for code-path where RX/TX callbacks are heavily involved. Signed-off-by: Konstantin Ananyev Reviewed-by: Ferruh Yigit Tested-by: Feifei Wang lib/ethdev/ethdev_private.c | 31 +++++ lib/ethdev/rte_ethdev.h | 270 +++++++++++++++++++++++++++++++------------- lib/ethdev/version.map | 3 + 3 files changed, 226 insertions(+), 78 deletions(-)