From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 43363326D for ; Tue, 4 Sep 2018 02:37:04 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id z96-v6so2092608wrb.8 for ; Mon, 03 Sep 2018 17:37:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=qH4+P/YCgq2+0Xv7UfJPJqg57NzOyDQaprPN9e6DOSA=; b=ktr4SrdBlc4fP1/Ti5kyecnO4BcQEPfiPVRpMVxjwsFTNopMm33GsrhLc/SGLgzNWB rmzFc5izJTwWKH6EAKHrJkaYE7W8jspstcuW2P/BY8YhLUy3AWsRaPXRbSWsS/WScEn9 SPywHd6JWYb++NtswFalMVTJlCra88kS+mqM4wWPAr6SihqDEDERPYu0Vo1Ix+IOWBGk QM90n46vEtz83bWN3A/BblkrAZP5k5KUbQxJ8OF3hMzq21V8bztoH56WzfhcUyVnridj uf7ngjQRj2hRaFi4xZXOOl54Sc5tZJGfgZkKbdnuJWYzta7bEPwyOwYx8rawW6MZj4TK j1pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=qH4+P/YCgq2+0Xv7UfJPJqg57NzOyDQaprPN9e6DOSA=; b=SNt8VBzU2Ur5j2dZlDZEgCRTdfy2WZanpiHH04qphFhmvbTJpRwMEuEAswi5B5Id4j YSQJf6BzdA4uXtwqPiUXovQCMPTqRaA3qDkIfEJJLnQ1AKrZmYIIPnSZGEPNH/MORq71 bjYy+QnctSiwlTGJvGLWqCqfJTOxSaEbrvPsbWAk/rcR4ca3ZFEbhEiyg8oAFOT3vYT2 aCkTmep7rHDMqRJlBu+WSOZ2XHAAdA/kqSi9MrwMlzftW8Vl145mfqm2071CNr+GgFmk 0O97obhQ4bExMq3tD2vhfxTfFeFU3QCkmtdIwdos1xRHwyyTwcilpKGb/P1JPp56FUg3 vQMQ== X-Gm-Message-State: APzg51Cgp1oKjDbiatMKAEWtZmbEjxfKcgaKv5cg9G8WApjQPkxawV+7 lOPinzb/zd3XhsmSdBBWOWy5FH6Yqbjh4FUbOOQ= X-Google-Smtp-Source: ANB0VdaGLo5fV8R8fiCBmZjM9gIhzyYsmoHPWCFvEdeglvAkdCcrjHxatMEywQ3S1JDfTLuZF+QlSUCoGeodHJB/qTQ= X-Received: by 2002:adf:e792:: with SMTP id n18-v6mr9449944wrm.136.1536021423716; Mon, 03 Sep 2018 17:37:03 -0700 (PDT) MIME-Version: 1.0 Sender: dan.gora@gmail.com Received: by 2002:adf:fbc1:0:0:0:0:0 with HTTP; Mon, 3 Sep 2018 17:36:23 -0700 (PDT) In-Reply-To: <611163de-bef7-488b-a77b-0e1ff190f1fb@intel.com> References: <20180628224513.18391-1-dg@adax.com> <20180629015508.26599-1-dg@adax.com> <20180629015508.26599-3-dg@adax.com> <611163de-bef7-488b-a77b-0e1ff190f1fb@intel.com> From: Dan Gora Date: Mon, 3 Sep 2018 21:36:23 -0300 X-Google-Sender-Auth: 8szfzHCh1Nw_8M_HK9tlgiuziNY Message-ID: To: Ferruh Yigit Cc: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2 02/10] kni: separate releasing netdev from freeing KNI interface X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2018 00:37:04 -0000 Hi Ferruh, I remembered now the motivation behind separating rte_kni_release() and rte_kni_free(). The problem is that the DPDK thread which calls rte_kni_release() _cannot_ be the same thread which handles callbacks from the KNI driver via rte_kni_handle_request(). This is because the thread which calls rte_kni_release() will be stuck down in ioctl(RTE_KNI_IOCTL_RELEASE) when the kernel calls the RTE_KNI_REQ_CFG_NETWORK_IF callback to the DPDK application. Since that thread cannot call rte_kni_handle_request(), the callback would then just timeout unless some other thread calls rte_kni_handle_request(). So then you are in a bit of a chicken and egg situation. You _have_ to have a separate thread calling rte_kni_handle_request periodically, but that thread also _cannot_ run after rte_kni_release returns (actually it's worse than that because it's actually after the ioctl(RTE_KNI_IOCTL_RELEASE) returns and the fifos are freed). So in order to resolve this, I separated the release from the freeing stages. This allows the DPDK application to keep the rte_kni_handle_request() thread running while rte_kni_release() is called so that it can handle the interface state callback, then kill that thread so that it cannot touch any 'struct rte_kni' resources, then free the struct rte_kni resources. thanks dan On Wed, Aug 29, 2018 at 7:59 AM, Ferruh Yigit wrote: >> When the kernel network interface is removed with unregister_netdev(), >> if the interface is up, it will generate a callback to mark the >> interface down, which calls kni_net_release(). kni_net_release() will >> block waiting for the DPDK application to call rte_kni_handle_request() >> to handle the callback, but it also needs the thread in the KNI driver >> (either the per-dev thread for multi-thread or the per-driver thread) >> to call kni_net_poll_resp() in order to wake the thread sleeping in >> kni_net_release (actually kni_net_process_request()). >> >> So now, KNI interfaces should be removed as such: >> >> 1) The user calls rte_kni_release(). This only unregisters the >> netdev in the kernel, but touches nothing else. This allows all the >> threads to run which are necessary to handle the callback into the >> DPDK application to mark the interface down. >> >> 2) The user stops the thread running rte_kni_handle_request(). >> After rte_kni_release() has been called, there will be no more >> callbacks for that interface so it is not necessary. It cannot be >> running at the same time that rte_kni_free() frees all of the FIFOs >> and DPDK memory for that KNI interface. >> >> 3) The user calls rte_kni_free(). This performs the RTE_KNI_IOCTL_FREE >> ioctl which calls kni_ioctl_free(). This function removes the struct >> kni_dev from the list of interfaces to poll (and kills the per-dev >> kthread, if configured for multi-thread), then frees the memory in >> the FIFOs.