From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id F0BD91BF13 for ; Fri, 15 Jun 2018 17:44:47 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Jun 2018 08:44:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,227,1526367600"; d="scan'208";a="57630472" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.66]) ([10.237.220.66]) by FMSMGA003.fm.intel.com with ESMTP; 15 Jun 2018 08:44:44 -0700 To: Qi Zhang , thomas@monjalon.net Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180607123849.14439-5-qi.z.zhang@intel.com> From: "Burakov, Anatoly" Message-ID: Date: Fri, 15 Jun 2018 16:44:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180607123849.14439-5-qi.z.zhang@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 04/22] ethdev: enable hotplug on multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jun 2018 15:44:48 -0000 On 07-Jun-18 1:38 PM, Qi Zhang wrote: > The patch introduce the solution to handle different hotplug cases in > multi-process situation, it include below scenario: > > 1. Attach a share device from primary > 2. Detach a share device from primary > 3. Attach a share device from secondary > 4. Detach a share device from secondary > 5. Attach a private device from secondary > 6. Detach a private device from secondary > 7. Detach a share device from secondary privately > 8. Attach a share device from secondary privately > > In primary-secondary process model, we assume device is shared by default. > that means attach or detach a device on any process will broadcast to > all other processes through mp channel then device information will be > synchronized on all processes. > > Any failure during attaching process will cause inconsistent status > between processes, so proper rollback action should be considered. > Also it is not safe to detach a share device when other process still use > it, so a handshake mechanism is introduced, it will be implemented in > following separate patch. > > Scenario for Case 1, 2: > > attach device > a) primary attach the new device if failed goto h). > b) primary send attach sync request to all secondary. > c) secondary receive request and attach device and send reply. > d) primary check the reply if all success go to i). > e) primary send attach rollback sync request to all secondary. > f) secondary receive the request and detach device and send reply. > g) primary receive the reply and detach device as rollback action. > h) attach fail > i) attach success > > detach device > a) primary perform pre-detach check, if device is locked, goto i). > b) primary send pre-detach sync request to all secondary. > c) secondary perform pre-detach check and send reply. > d) primary check the reply if any fail goto i). > e) primary send detach sync request to all secondary > f) secondary detach the device and send reply (assume no fail) > g) primary detach the device. > h) detach success > i) detach failed > > Case 3, 4: > This will be implemented in following patch. > > Case 5, 6: > Secondary process can attach private device which only visible to itself, > in this case no IPC is involved, primary process is not allowd to have > private device so far. > > Case 7, 8: > Secondary process can also temporally to detach a share device "privately" > then attach it back later, this action also not impact other processes. > > APIs chenages: > > rte_eth_dev_attach and rte_eth_dev_attach are extended to support > share device attach/detach in primary-secondary process model, it will > be called in case 1,2,3,4. > > New API rte_eth_dev_attach_private and rte_eth_dev_detach_private are > introduced to cover case 5,6,7,8, this API can only be invoked in secondary > process. > > Signed-off-by: Qi Zhang > --- > lib/librte_eal/common/eal_private.h | 8 ++ > lib/librte_eal/linuxapp/eal/eal.c | 6 ++ > lib/librte_ethdev/Makefile | 1 + > lib/librte_ethdev/rte_ethdev.c | 183 ++++++++++++++++++++++++++++--- > lib/librte_ethdev/rte_ethdev.h | 37 +++++++ > lib/librte_ethdev/rte_ethdev_core.h | 5 + > lib/librte_ethdev/rte_ethdev_driver.h | 27 +++++ > lib/librte_ethdev/rte_ethdev_mp.c | 195 ++++++++++++++++++++++++++++++++++ > lib/librte_ethdev/rte_ethdev_mp.h | 44 ++++++++ > 9 files changed, 489 insertions(+), 17 deletions(-) > create mode 100644 lib/librte_ethdev/rte_ethdev_mp.c > create mode 100644 lib/librte_ethdev/rte_ethdev_mp.h > Haven't looked at the code yet, but general comment: please don't prefix internal-only files with rte_, it makes it look like they are part of external API. -- Thanks, Anatoly