From: Dan Gora <dg@adax.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v2 02/10] kni: separate releasing netdev from freeing KNI interface
Date: Mon, 3 Sep 2018 21:36:23 -0300 [thread overview]
Message-ID: <CAGyogRaKnRNQAfne28u5dENkbSiEou-ZSi0UwNGquHHb2+hhjA@mail.gmail.com> (raw)
In-Reply-To: <611163de-bef7-488b-a77b-0e1ff190f1fb@intel.com>
Hi Ferruh,
I remembered now the motivation behind separating rte_kni_release()
and rte_kni_free().
The problem is that the DPDK thread which calls rte_kni_release()
_cannot_ be the same thread which handles callbacks from the KNI
driver via rte_kni_handle_request(). This is because the thread which
calls rte_kni_release() will be stuck down in
ioctl(RTE_KNI_IOCTL_RELEASE) when the kernel calls the
RTE_KNI_REQ_CFG_NETWORK_IF callback to the DPDK application. Since
that thread cannot call rte_kni_handle_request(), the callback would
then just timeout unless some other thread calls
rte_kni_handle_request().
So then you are in a bit of a chicken and egg situation. You _have_
to have a separate thread calling rte_kni_handle_request periodically,
but that thread also _cannot_ run after rte_kni_release returns
(actually it's worse than that because it's actually after the
ioctl(RTE_KNI_IOCTL_RELEASE) returns and the fifos are freed).
So in order to resolve this, I separated the release from the freeing
stages. This allows the DPDK application to keep the
rte_kni_handle_request() thread running while rte_kni_release() is
called so that it can handle the interface state callback, then kill
that thread so that it cannot touch any 'struct rte_kni' resources,
then free the struct rte_kni resources.
thanks
dan
On Wed, Aug 29, 2018 at 7:59 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> When the kernel network interface is removed with unregister_netdev(),
>> if the interface is up, it will generate a callback to mark the
>> interface down, which calls kni_net_release(). kni_net_release() will
>> block waiting for the DPDK application to call rte_kni_handle_request()
>> to handle the callback, but it also needs the thread in the KNI driver
>> (either the per-dev thread for multi-thread or the per-driver thread)
>> to call kni_net_poll_resp() in order to wake the thread sleeping in
>> kni_net_release (actually kni_net_process_request()).
>>
>> So now, KNI interfaces should be removed as such:
>>
>> 1) The user calls rte_kni_release(). This only unregisters the
>> netdev in the kernel, but touches nothing else. This allows all the
>> threads to run which are necessary to handle the callback into the
>> DPDK application to mark the interface down.
>>
>> 2) The user stops the thread running rte_kni_handle_request().
>> After rte_kni_release() has been called, there will be no more
>> callbacks for that interface so it is not necessary. It cannot be
>> running at the same time that rte_kni_free() frees all of the FIFOs
>> and DPDK memory for that KNI interface.
>>
>> 3) The user calls rte_kni_free(). This performs the RTE_KNI_IOCTL_FREE
>> ioctl which calls kni_ioctl_free(). This function removes the struct
>> kni_dev from the list of interfaces to poll (and kills the per-dev
>> kthread, if configured for multi-thread), then frees the memory in
>> the FIFOs.
next prev parent reply other threads:[~2018-09-04 0:37 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-28 22:45 [dpdk-dev] [PATCH 00/10] kni: Interface detach and link status fixes Dan Gora
2018-06-29 1:54 ` Dan Gora
2018-06-29 1:54 ` [dpdk-dev] [PATCH v2 01/10] kni: remove unused variables from struct kni_dev Dan Gora
2018-08-29 10:29 ` Ferruh Yigit
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 02/10] kni: separate releasing netdev from freeing KNI interface Dan Gora
2018-08-29 10:59 ` Ferruh Yigit
2018-09-04 0:20 ` Dan Gora
2018-09-04 0:36 ` Dan Gora [this message]
2018-10-10 17:24 ` Ferruh Yigit
2018-10-10 18:18 ` Dan Gora
2018-10-10 22:51 ` Ferruh Yigit
2018-10-10 23:38 ` Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 03/10] kni: don't touch struct kni_dev after freeing Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 04/10] kni: add rte_kni_free to KNI library Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 05/10] kni: don't run rte_kni_handle_request after interface release Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 06/10] kni: increase length of timeout for KNI responses Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 07/10] kni: update kni test for rte_kni_free Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 08/10] kni: add rte_kni_free to KNI example app Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 09/10] kni: add rte_kni_free to KNI vdev driver Dan Gora
2018-06-29 1:55 ` [dpdk-dev] [PATCH v2 10/10] kni: add API to set link status on kernel interface Dan Gora
2018-08-29 11:48 ` Ferruh Yigit
2018-08-29 21:10 ` Dan Gora
2018-08-29 22:01 ` Stephen Hemminger
2018-08-29 15:54 ` Stephen Hemminger
2018-08-29 21:02 ` Dan Gora
2018-08-29 22:00 ` Stephen Hemminger
2018-08-29 22:12 ` Dan Gora
2018-08-29 22:41 ` Dan Gora
2018-08-29 23:10 ` Stephen Hemminger
2018-08-30 9:49 ` Igor Ryzhov
2018-08-30 10:32 ` Igor Ryzhov
2018-08-30 21:41 ` Dan Gora
2018-08-30 22:09 ` Stephen Hemminger
2018-08-30 22:11 ` Dan Gora
2018-09-04 0:47 ` Dan Gora
2018-09-05 12:57 ` Stephen Hemminger
2018-09-11 21:45 ` Dan Gora
2018-09-11 21:52 ` Stephen Hemminger
2018-09-11 22:07 ` Dan Gora
2018-09-11 23:14 ` Stephen Hemminger
2018-09-12 4:02 ` Jason Wang
2018-09-11 23:14 ` [dpdk-dev] [PATCH 0/2] " Dan Gora
2018-09-11 23:14 ` [dpdk-dev] [PATCH 1/2] " Dan Gora
2018-09-11 23:18 ` Dan Gora
2018-07-20 11:36 ` [dpdk-dev] [PATCH 00/10] kni: Interface detach and link status fixes Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAGyogRaKnRNQAfne28u5dENkbSiEou-ZSi0UwNGquHHb2+hhjA@mail.gmail.com \
--to=dg@adax.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).