DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ferruh Yigit <ferruh.yigit@xilinx.com>
To: Lewis Donzis <lew@perftech.com>,
	Konstantin Ananyev <konstantin.ananyev@intel.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>, dev <dev@dpdk.org>,
	"Wang,  Yong" <yongwang@vmware.com>,
	Jochen Behrens <jbehrens@vmware.com>
Subject: Re: vmxnet3 no longer functional on DPDK 21.11
Date: Fri, 3 Jun 2022 16:25:04 +0100	[thread overview]
Message-ID: <85b37945-914f-dec6-f56d-442f900ca5b5@xilinx.com> (raw)
In-Reply-To: <1796133353.5470017.1654262374494.JavaMail.zimbra@donzis.com>

On 6/3/2022 2:19 PM, Lewis Donzis wrote:
> Hi, all.
> 
> Resurrecting this thread from six months ago, I apologize for not having more time to dig into it, but in light of recent findings, I see numerous other drivers and other parts of the code that have comments to the effect that "FreeBSD doesn't support interrupts" and they effectively #ifdef out the attempt.
> 
> Could this be as simple as needing to do the same in vmxnet3?  Empirically, ignoring the error from rte_intr_enable() allows the driver to work normally, for what that's worth.
> 
> Thanks,
> lew

cc'ed new maintainer, Jochen.

> 
> ----- On Dec 6, 2021, at 6:08 AM, Konstantin Ananyev konstantin.ananyev@intel.com wrote:
> 
>>> -----Original Message-----
>>> From: Richardson, Bruce <bruce.richardson@intel.com>
>>> Sent: Monday, December 6, 2021 9:17 AM
>>> To: Lewis Donzis <lew@perftech.com>
>>> Cc: dev <dev@dpdk.org>; Wang, Yong <yongwang@vmware.com>; Ananyev, Konstantin
>>> <konstantin.ananyev@intel.com>
>>> Subject: Re: vmxnet3 no longer functional on DPDK 21.11
>>>
>>> On Sun, Dec 05, 2021 at 07:52:33PM -0600, Lewis Donzis wrote:
>>>>
>>>>
>>>> ----- On Nov 30, 2021, at 7:42 AM, Bruce Richardson bruce.richardson@intel.com
>>>> wrote:
>>>>
>>>>> On Mon, Nov 29, 2021 at 02:45:15PM -0600, Lewis Donzis wrote:
>>>>>>     Hello.
>>>>>>     We just upgraded from 21.08 to 21.11 and it's rather astounding the
>>>>>>     number of incompatible changes in three months.  Not a big deal, just
>>>>>>     kind of a surprise, that's all.
>>>>>>     Anyway, the problem is that the vmxnet3 driver is no longer functional
>>>>>>     on FreeBSD.
>>>>>>     In drivers/net/vmxnet3/vmxnet3_ethdev.c, vmxnet3_dev_start() gets an
>>>>>>     error calling rte_intr_enable().  So it logs "interrupt enable failed"
>>>>>>     and returns an error.
>>>>>>     In lib/eal/freebsd/eal_interrupts.c, rte_intr_enable() is returning an
>>>>>>     error because rte_intr_dev_fd_get(intr_handle) is returning -1.
>>>>>>     I don't see how that could ever return anything other than -1 since it
>>>>>>     appears that there is no code that ever calls rte_intr_dev_fd_set()
>>>>>>     with a value other than -1 on FreeBSD.  Also weird to me is that even
>>>>>>     if it didn't get an error, the switch statement that follows looks like
>>>>>>     it will return an error in every case.
>>>>>>     Nonetheless, it worked in 21.08, and I can't quite see why the
>>>>>>     difference, so I must be missing something.
>>>>>>     For the moment, I just commented the "return -EIO" in vmxnet3_ethdev.c,
>>>>>>     and it's now working again, but that's obviously not the correct
>>>>>>     solution.
>>>>>>     Can someone who's knowledgable about this mechanism perhaps explain a
>>>>>>     little bit about what's going on?  I'll be happy to help troubleshoot.
>>>>>>     It seems like it must be something simple, but I just don't see it yet.
>>>>>
>>>>> Hi
>>>>>
>>>>> if you have the chance, it would be useful if you could use "git bisect" to
>>>>> identify the commit in 21.11 that broke this driver. Looking through the
>>>>> logs for 21.11 I can't identify any particular likely-looking commit, so
>>>>> bisect is likely a good way to start looking into this.
>>>>>
>>>>> Regards,
>>>>> /Bruce
>>>>
>>>> Hi, Bruce.  git bisect is very time-consuming and very cool!
>>>>
>>>> I went back to 21.08, about 1100 commits, and worked through the process, but
>>>> then I realized that I had forgotten to run ninja on one of
>>> the steps, so I did it again.
>>>>
>>>> I also re-checked it after the bisect, just to make sure that
>>>> c87d435a4d79739c0cec2ed280b94b41cb908af7 is good, and
>>> 7a0935239b9eb817c65c03554a9954ddb8ea5044 is bad.
>>>>
>>>> Thanks,
>>>> lew
>>>>
>>>
>>> Many thanks for taking the time to do this. Adding Konstantin to thread as
>>> author of the commit you identified. Konstantin, any thoughts on this
>>> issue?
>>
>> Hmm, that's looks really strange to me.
>> So to clarify, it fails at:
>> static int
>> vmxnet3_dev_start(struct rte_eth_dev *dev)
>> {
>> 	...
>> line 1695:
>> 	if (rte_intr_enable(dev->intr_handle) < 0) {
>>                 PMD_INIT_LOG(ERR, "interrupt enable failed");
>>                 return -EIO;
>>         }
>>
>> Right?
>>
>> The strange thing here is that 7a0935239b9e
>> doesn't change dev_start or rte_intr code in any way.
>> All it does - change rte_eth_rx_burst/rte_eth_tx_burst and other fast-path
>> functions.
>> Anyway, if git blames that commit, let's try to figure out what is going on.
>> Unfortunately, I don't have freebsd with vmxnet3, so will need to rely on your
>> help here.
>> As the first thing can you try to run testpmd build with last good commit
>> (c87d435a4d79)
>> and then testpmd build with bad commit applied and collect for both cases:
>> - contents of 'struct rte_eth_dev' and ' rte_eth_dev->intr_handle' for
>>   your vmxnet3 port
>> - debug log output (--log-level=eal,debug --log-level=pmd,debug)
>>
>> Konstantin


  reply	other threads:[~2022-06-03 15:25 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-29 20:45 Lewis Donzis
2021-11-30  8:53 ` Ferruh Yigit
2021-11-30 13:42 ` Bruce Richardson
2021-12-06  1:52   ` Lewis Donzis
2021-12-06  9:16     ` Bruce Richardson
2021-12-06 12:08       ` Ananyev, Konstantin
2021-12-06 13:58         ` Lewis Donzis
2022-06-03 13:19         ` Lewis Donzis
2022-06-03 15:25           ` Ferruh Yigit [this message]
2024-01-06 14:50           ` Lewis Donzis
2024-01-09 10:21             ` Bruce Richardson
2024-01-09 13:46               ` Lewis Donzis
2024-01-09 14:28                 ` Bruce Richardson
2024-01-09 15:21                   ` Lewis Donzis
2024-01-09 15:35                     ` Bruce Richardson
2024-01-09 23:55               ` Stephen Hemminger
2024-01-10 13:36                 ` Lewis Donzis
2024-01-09 14:23             ` [PATCH] net/vmxnet3: fix use of interrupts on FreeBSD Bruce Richardson
2024-01-09 16:00               ` Lewis Donzis
2024-01-11 12:03                 ` Ferruh Yigit
2024-01-24 12:34                   ` Lewis Donzis
2024-01-24 13:58                     ` Ferruh Yigit
2024-01-24 14:04                       ` Lewis Donzis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=85b37945-914f-dec6-f56d-442f900ca5b5@xilinx.com \
    --to=ferruh.yigit@xilinx.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=jbehrens@vmware.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=lew@perftech.com \
    --cc=yongwang@vmware.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).