patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Igor Ryzhov <iryzhov@nfware.com>
To: Stephen Hemminger <stephen@networkplumber.org>
Cc: Ferruh Yigit <ferruh.yigit@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>, dev <dev@dpdk.org>,
	 dpdk stable <stable@dpdk.org>
Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH] kni: fix kernel deadlock when using mlx devices
Date: Tue, 28 Jul 2020 11:56:26 +0300	[thread overview]
Message-ID: <CAF+s_FxF_s3ETF2Ewt-_nPDEXyg0zBroNyL-Rz0W8mX883bsZQ@mail.gmail.com> (raw)
In-Reply-To: <20200727105255.74981391@hermes.lan>

Hi,

Didn't see this patch previously, but we came up with the same idea
internally and also faced a hang during the application shutdown.
We didn't dig deep, but it occurred in kni_release function.

Igor

On Mon, Jul 27, 2020 at 8:53 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Mon, 27 Jul 2020 18:33:08 +0100
> Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> > On 5/6/2020 1:14 AM, Stephen Hemminger wrote:
> > > On Wed, 18 Mar 2020 16:17:57 +0100
> > > Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > >> 17/01/2020 17:43, Ferruh Yigit:
> > >>> On 12/22/2019 5:55 PM, Stephen Hemminger wrote:
> > >>>> This fixes a deadlock when using KNI with bifurcated drivers.
> > >>>> Bringing kni device up always times out when using Mellanox
> > >>>> devices.
> > >>>>
> > >>>> The kernel KNI driver sends message to userspace to complete
> > >>>> the request. For the case of bifurcated driver, this may involve
> > >>>> an additional request to kernel to change state. This request
> > >>>> would deadlock because KNI was holding the RTNL mutex.
> > >>>>
> > >>>> This was a bad design which goes back to the original code.
> > >>>> A workaround is for KNI driver to drop RTNL while waiting.
> > >>>> To prevent the device from disappearing while the operation
> > >>>> is in progress, it needs to hold reference to network device
> > >>>> while waiting.
> > >>>>
> > >>>> As an added benefit, an useless error check can also be removed.
> > >>>>
> > >>>> Fixes: 3fc5ca2f6352 ("kni: initial import")
> > >>>> Cc: stable@dpdk.org
> > >>>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > >>>> ---
> > >>>
> > >>> This patch cause a hang on my server, not sure what exactly was the
> problem but
> > >>> kernel log was continuously printing "Cannot send to req_q". Will
> dig more.
> > >>
> > >> Ferruh, did you have a chance to check what is hanging?
> > >> Stephen, is there any news on your side?
> > >>
> > >>
> > >
> > > It did not hang when I tested it. The bug report is still open
> > >
> >
> > Sorry for the delay, since I am working remotely I was worried about
> loosing the
> > connection to my server, finally I did create a virtual environment to
> test again.
> >
> > I confirm the hang observed %100 when two different process updates the
> kni
> > interface, like two different process sets the mtu. Without this patch
> this
> > works fine.
> >
> > I understand the motivation of the patch, but with change there is a
> possibility
> > to hang the server, which we can't allow, need to find another way. Can
> updating
> > mlx interface wait KNI interface operation to complete?
>
> Still KNI driver is broken. Calling userspace with RTNL held is
> fundamentally
> broken design. If KNI were to be incorporated in upstream kernel, then the
> netdev
> developer would see this.
>
> What ever solution you think is best.
> I will continue to recommend against anyone using KNI.
>

      reply	other threads:[~2020-07-28  8:56 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-22 17:55 [dpdk-stable] " Stephen Hemminger
2020-01-17 16:43 ` Ferruh Yigit
2020-03-18 15:17   ` [dpdk-stable] [dpdk-dev] " Thomas Monjalon
2020-05-06  0:14     ` Stephen Hemminger
2020-07-27 17:33       ` Ferruh Yigit
2020-07-27 17:52         ` Stephen Hemminger
2020-07-28  8:56           ` Igor Ryzhov [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAF+s_FxF_s3ETF2Ewt-_nPDEXyg0zBroNyL-Rz0W8mX883bsZQ@mail.gmail.com \
    --to=iryzhov@nfware.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=stable@dpdk.org \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).