Re: vmxnet3 no longer functional on DPDK 21.11

[Lewis Donzis <lew@perftech.com>]

----- On Nov 30, 2021, at 7:42 AM, Bruce Richardson bruce.richardson@intel.com wrote:

> On Mon, Nov 29, 2021 at 02:45:15PM -0600, Lewis Donzis wrote:
>>    Hello.
>>    We just upgraded from 21.08 to 21.11 and it's rather astounding the
>>    number of incompatible changes in three months.  Not a big deal, just
>>    kind of a surprise, that's all.
>>    Anyway, the problem is that the vmxnet3 driver is no longer functional
>>    on FreeBSD.
>>    In drivers/net/vmxnet3/vmxnet3_ethdev.c, vmxnet3_dev_start() gets an
>>    error calling rte_intr_enable().  So it logs "interrupt enable failed"
>>    and returns an error.
>>    In lib/eal/freebsd/eal_interrupts.c, rte_intr_enable() is returning an
>>    error because rte_intr_dev_fd_get(intr_handle) is returning -1.
>>    I don't see how that could ever return anything other than -1 since it
>>    appears that there is no code that ever calls rte_intr_dev_fd_set()
>>    with a value other than -1 on FreeBSD.  Also weird to me is that even
>>    if it didn't get an error, the switch statement that follows looks like
>>    it will return an error in every case.
>>    Nonetheless, it worked in 21.08, and I can't quite see why the
>>    difference, so I must be missing something.
>>    For the moment, I just commented the "return -EIO" in vmxnet3_ethdev.c,
>>    and it's now working again, but that's obviously not the correct
>>    solution.
>>    Can someone who's knowledgable about this mechanism perhaps explain a
>>    little bit about what's going on?  I'll be happy to help troubleshoot.
>>    It seems like it must be something simple, but I just don't see it yet.
> 
> Hi
> 
> if you have the chance, it would be useful if you could use "git bisect" to
> identify the commit in 21.11 that broke this driver. Looking through the
> logs for 21.11 I can't identify any particular likely-looking commit, so
> bisect is likely a good way to start looking into this.
> 
> Regards,
> /Bruce

Hi, Bruce.  git bisect is very time-consuming and very cool!

I went back to 21.08, about 1100 commits, and worked through the process, but then I realized that I had forgotten to run ninja on one of the steps, so I did it again.

I also re-checked it after the bisect, just to make sure that c87d435a4d79739c0cec2ed280b94b41cb908af7 is good, and 7a0935239b9eb817c65c03554a9954ddb8ea5044 is bad.

Thanks,
lew


Here's the result:

root@fbdev:/usr/local/share/dpdk-git # git bisect start
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
root@fbdev:/usr/local/share/dpdk-git # git bisect good 74bd4072996e64b0051d24d8d641554d225db196
Bisecting: 556 revisions left to test after this (roughly 9 steps)
[e2a289a788c0a128a15bc0f1099af7c031201ac5] net/ngbe: add mailbox process operations
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
Bisecting: 277 revisions left to test after this (roughly 8 steps)
[5906be5af6570db8b70b307c96aace0b096d1a2c] ethdev: fix ID spelling in comments and log messages
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
Bisecting: 138 revisions left to test after this (roughly 7 steps)
[a7c236b894a848c7bb9afb773a7e3c13615abaa8] net/cnxk: support meter ops get
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
Bisecting: 69 revisions left to test after this (roughly 6 steps)
[14fc81aed73842d976dd19a93ca47e22d61c1759] ethdev: update modify field flow action
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
Bisecting: 34 revisions left to test after this (roughly 5 steps)
[cdea571becb4dabf9962455f671af0c99594e380] common/sfc_efx/base: add flag to use Rx prefix user flag
root@fbdev:/usr/local/share/dpdk-git # git bisect good
Bisecting: 17 revisions left to test after this (roughly 4 steps)
[7a0935239b9eb817c65c03554a9954ddb8ea5044] ethdev: make fast-path functions to use new flat array
root@fbdev:/usr/local/share/dpdk-git # git bisect bad
Bisecting: 8 revisions left to test after this (roughly 3 steps)
[012bf708c20f4b23d055717e28f8de74887113d8] net/sfc: support group flows in tunnel offload
root@fbdev:/usr/local/share/dpdk-git # git bisect good
Bisecting: 4 revisions left to test after this (roughly 2 steps)
[9df2d8f5cc9653d6413cb2240c067ea455ab7c3c] net/sfc: support counters in tunnel offload jump rules
root@fbdev:/usr/local/share/dpdk-git # git bisect good
Bisecting: 2 revisions left to test after this (roughly 1 step)
[c024496ae8c8c075b0d0a3b43119475787b24b45] ethdev: allocate max space for internal queue array
root@fbdev:/usr/local/share/dpdk-git # git bisect good
Bisecting: 0 revisions left to test after this (roughly 1 step)
[c87d435a4d79739c0cec2ed280b94b41cb908af7] ethdev: copy fast-path API into separate structure
root@fbdev:/usr/local/share/dpdk-git # git bisect good
7a0935239b9eb817c65c03554a9954ddb8ea5044 is the first bad commit
commit 7a0935239b9eb817c65c03554a9954ddb8ea5044
Author: Konstantin Ananyev <konstantin.ananyev@intel.com>
Date:   Wed Oct 13 14:37:02 2021 +0100

    ethdev: make fast-path functions to use new flat array
    
    Rework fast-path ethdev functions to use rte_eth_fp_ops[].
    While it is an API/ABI breakage, this change is intended to be
    transparent for both users (no changes in user app is required) and
    PMD developers (no changes in PMD is required).
    One extra thing to note - RX/TX callback invocation will cause extra
    function call with these changes. That might cause some insignificant
    slowdown for code-path where RX/TX callbacks are heavily involved.
    
    Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
    Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
    Tested-by: Feifei Wang <feifei.wang2@arm.com>

 lib/ethdev/ethdev_private.c |  31 +++++
 lib/ethdev/rte_ethdev.h     | 270 +++++++++++++++++++++++++++++++-------------
 lib/ethdev/version.map      |   3 +
 3 files changed, 226 insertions(+), 78 deletions(-)