From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A0741A046B for ; Fri, 26 Jul 2019 11:54:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A622A1C40D; Fri, 26 Jul 2019 11:54:03 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id D3DB01C406 for ; Fri, 26 Jul 2019 11:54:01 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jul 2019 02:54:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,310,1559545200"; d="scan'208";a="369481993" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.252.25.220]) ([10.252.25.220]) by fmsmga005.fm.intel.com with ESMTP; 26 Jul 2019 02:53:59 -0700 To: "Ananyev, Konstantin" , "Richardson, Bruce" Cc: "dev@dpdk.org" , Thomas Monjalon References: <29cf5458-8459-0187-13b1-44277283fc93@intel.com> <20190726093951.GA1629@bricha3-MOBL.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580168A5BA83@irsmsx105.ger.corp.intel.com> From: "Burakov, Anatoly" Message-ID: <06bb50da-e56f-3bf0-920f-001480150f5d@intel.com> Date: Fri, 26 Jul 2019 10:53:58 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <2601191342CEEE43887BDE71AB9772580168A5BA83@irsmsx105.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] Should we disallow running secondaries after primary has died? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 26-Jul-19 10:50 AM, Ananyev, Konstantin wrote: > > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson >> Sent: Friday, July 26, 2019 10:40 AM >> To: Burakov, Anatoly >> Cc: dev@dpdk.org; Thomas Monjalon >> Subject: Re: [dpdk-dev] Should we disallow running secondaries after primary has died? >> >> On Fri, Jul 26, 2019 at 10:05:02AM +0100, Burakov, Anatoly wrote: >>> Hi all, >>> >>> While investigating this bug: >>> >>> https://bugs.dpdk.org/show_bug.cgi?id=284 >>> >>> I came across a realization that, when primary process dies, very little >>> actually works. There are some documented issues that are already present >>> when secondary processes keep running, like memory map becoming static, and >>> hotplug not working any more. >>> >>> What is less known (and documented) is that VFIO also completely stops >>> working when initializing processes, because some time since 18.xx releases, >>> we've fixed a long standing VFIO-related bug that had to do with creating >>> new containers every time a secondary is run - secondary processes will now >>> reuse the primary process's container instead. >>> >>> Meaning, for VFIO devices, secondary process *initialization* will fail >>> after primary process has died, because there is no longer a process from >>> which we can get the VFIO container from. Things will still sort-of work >>> with igb_uio or in vfio-noiommu mode, but again - no memory map updates, no >>> hotplug, potentially other things that i don't even know about. >>> >>> Therefore, while ideally we would like people to have primary process always >>> running, the least we can do to avoid documenting a complex matrix of "what >>> is supported in which case" is to disallow secondary process initialization >>> after primary process has died. >>> >>> ("disallow" as in "explicitly document it as unsupported", although we can >>> also outright prevent it if we want - rte_eal_primary_proc_alive() will tell >>> us that) >>> >> Documenting this limitation seems a good thing to do. I'm not sure that >> it's worthwhile trying to make the scenario (of running a secondary after a >> primary has terminated) supported. >> >> /Bruce > > NP to disallow it. > In fact, I think it would be easier for everyone just to drop current DPDK MP model, > and keep just standalone DPDK instances. That's the dream, but i don't think it'll ever come to fruition, at least not without a huge push from the community. -- Thanks, Anatoly