From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D7F221B3B1 for ; Thu, 13 Dec 2018 02:22:45 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Dec 2018 17:22:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,346,1539673200"; d="scan'208";a="101112804" Received: from dpdk-xiao-1.sh.intel.com ([10.67.111.145]) by orsmga008.jf.intel.com with ESMTP; 12 Dec 2018 17:22:42 -0800 From: Xiao Wang To: alejandro.lucero@netronome.com, tiwei.bie@intel.com Cc: maxime.coquelin@redhat.com, dev@dpdk.org, zhihong.wang@intel.com, xiaolong.ye@intel.com, Xiao Wang Date: Thu, 13 Dec 2018 09:10:05 +0800 Message-Id: <20181213011014.110089-1-xiao.w.wang@intel.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20181128094607.106173-3-xiao.w.wang@intel.com> References: <20181128094607.106173-3-xiao.w.wang@intel.com> Subject: [dpdk-dev] [PATCH v2 0/9] support SW assisted VDPA live migration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 01:22:46 -0000 In the previous VDPA implementation we have enabled live migration support by HW accelerator doing all the stuff, including dirty page logging and device status report/restore. In this mode VDPA sample daemon and device driver just takes care of the control path and does not involve in data path, so there's almost 0 CPU resource usage. This mode requires device to have dirty page logging capability. This patch series adds live migration support for devices without logging capability. VDPA driver could set up a relay thread standing between the guest and device when live migration happens, this relay intervenes into the communication between guest virtio driver and physical virtio accelerator, it helps device to do a vring relay and passingly log dirty pages. Thus some CPU resource will be consumed in this scenario, percentage depending on the network throughput. Some new helpers are added into vhost lib for this VDPA SW fallback: - rte_vhost_host_notifier_ctrl, to enable/disable the VDPA direct-IO datapath. - rte_vdpa_relay_avail_ring, to relay the available ring from guest vring to mediate vring. - rte_vdpa_relay_used_ring, to relay the used ring from mediate vring to guest vring. Some existing helpers are also leveraged for SW fallback setup, like VFIO interrupt configuration, IOMMU table programming, etc. This patch enables this SW assisted VDPA live migration in ifc driver. Since ifcvf also supports HW dirty page logging, we add a new devarg for user to select if the SW mode is used or not. v2: * Reword the vdpa host notifier control API comment. * Make the vring relay API parameter as "void *" to accomodate the future potential new ring layout, e.g. packed ring. * Add parameter check for the new API. * Add memory barrier for ring idx update. * Remove the used ring logging in the relay. * Some comment fix and code cleaning according to Tiwei's comment. * Add release note update. Xiao Wang (9): vhost: provide helper for host notifier ctrl vhost: provide helpers for virtio ring relay net/ifc: dump debug message for error net/ifc: store only registered device instance net/ifc: detect if VDPA mode is specified net/ifc: add devarg for LM mode net/ifc: use lib API for used ring logging net/ifc: support SW assisted VDPA live migration doc: update ifc NIC document doc/guides/nics/ifc.rst | 7 + doc/guides/rel_notes/release_19_02.rst | 5 + drivers/net/ifc/base/ifcvf.h | 1 + drivers/net/ifc/ifcvf_vdpa.c | 461 ++++++++++++++++++++++++++++++--- lib/librte_vhost/rte_vdpa.h | 56 ++++ lib/librte_vhost/rte_vhost_version.map | 3 + lib/librte_vhost/vdpa.c | 187 +++++++++++++ lib/librte_vhost/vhost.c | 3 +- lib/librte_vhost/vhost.h | 40 +++ lib/librte_vhost/vhost_user.c | 7 +- lib/librte_vhost/virtio_net.c | 39 --- 11 files changed, 731 insertions(+), 78 deletions(-) -- 2.15.1