From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A176542DBE; Mon, 3 Jul 2023 07:15:12 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32E7940EF0; Mon, 3 Jul 2023 07:15:12 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 46DEE40156 for ; Mon, 3 Jul 2023 07:15:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688361310; x=1719897310; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=rf0X1OgGS7SHd3aWejaI9C5nUZTqBl+vLGEltI9oWKU=; b=SpKcmd2uqqQ4aEd207Q6UyWSVlyaF6eAa5PWlABCc/EWxDw4v74fTFJI bh5IjTG4goeg3W/9Ll0OnyQKTjo2QTFt/mcrHD3r0suMQIkQbosUPNkZD A0ukdb2UvKiNeoxSZvimsLMNeLXw4qISXy1ubJ+eJ8xVWcz9uM9+1o865 q7CpzvF1Z7NLmgfsQoWpjTovGqYVir/cd2yLSFU/Lm6l07drVmX84aDEf 6oj7WDqAftOZtUhQRrxiEepsGCko7foEhVrEpHMGPEnycdz1Nn1EQQgKY jCVTwHVk/t5mWN0zMl2aFCmJ0okIMpg5PYIMFBXqYMF1M7PSBz+FBkuJV w==; X-IronPort-AV: E=McAfee;i="6600,9927,10759"; a="366276331" X-IronPort-AV: E=Sophos;i="6.01,177,1684825200"; d="scan'208";a="366276331" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jul 2023 22:15:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10759"; a="721644312" X-IronPort-AV: E=Sophos;i="6.01,177,1684825200"; d="scan'208";a="721644312" Received: from dpdk-liulingy-2.sh.intel.com ([10.67.119.34]) by fmsmga007.fm.intel.com with ESMTP; 02 Jul 2023 22:15:07 -0700 From: Lingyu Liu To: dev@dpdk.org, yahui.cao@intel.com, qi.z.zhang@intel.com, beilei.xing@intel.com Cc: Lingyu Liu Subject: [PATCH] doc: add ice VF live migration guide Date: Mon, 3 Jul 2023 05:14:49 +0000 Message-Id: <20230703051449.191259-1-lingyu.liu@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Describe VF live migration steps for Intel E810 device. Signed-off-by: Lingyu Liu --- doc/guides/nics/intel_vf.rst | 93 ++++++++++++++++++++++++++ doc/guides/rel_notes/release_23_07.rst | 3 + 2 files changed, 96 insertions(+) diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst index d365dbc185..b0aed5f5a5 100644 --- a/doc/guides/nics/intel_vf.rst +++ b/doc/guides/nics/intel_vf.rst @@ -622,6 +622,99 @@ which belongs to the destination VF on the VM. Inter-VM Communication +Live Migrating a VM running DPDK +-------------------------------- + +The following describes a target environment: + +* Host Operating System: Ubuntu 20.04.5 + +* Guest Operating System: Ubuntu 20.04.5 + +* Linux Kernel Version: 5.15.0-72-generic + +* Target Applications: dpdk-testpmd + +* Ice Kernel Driver Version: 1.11.17.1 ``_ + +* Qemu Version: 7.2 + +The setup procedure is as follows: + +#. Before booting the Host OS, open **BIOS setup** and enable **Intel® VT features**. + +#. In the Host OS + + Install the ice driver and migration driver: + + .. code-block:: console + + insmod ice.ko + insmod ice-vfio-pci.ko + + Create 2 VFs and bind them to vfio pci driver: + + .. code-block:: console + + echo 2 > /sys/bus/pci/devices/0000:ca:00.1/sriov_numvfs + echo "8086 1889" > /sys/bus/pci/drivers/ice-vfio-pci/new_id + dpdk-devbind.py -b ice-vfio-pci 0000:ca:11.0 + dpdk-devbind.py -b ice-vfio-pci 0000:ca:11.1 + + .. note:: + + The command above creates two vfs for device 0000:ca:00.1: + + .. code-block:: console + + 0000:ca:11.0 'Ethernet Adaptive Virtual Function 1889' if= drv=ice-vfio-pci unused=iavf + 0000:ca:11.1 'Ethernet Adaptive Virtual Function 1889' if= drv=ice-vfio-pci unused=iavf + +#. Now, start the migration source Virtual Machine by running the following command: + + .. code-block:: console + + qemu/build/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu host -m 4G -smp 1 -device vfio-pci,host=0000:ca:11.0,x-enable-migration=true,x-pre-copy-dirty-page-tracking=off -drive file=ubuntu-2004.qcow2 -nic user,hostfwd=tcp::5555-:22 -monitor stdio + + .. note:: + The vfio-pci,host=0000:ca:11.0 value indicates that you want to attach a vfio PCI device + to a Virtual Machine and the respective (Bus:Device.Function) numbers should be passed, + x-enable-migration=true indicates that this VF supports migration. Dirty page tracking + is not supported, so set x-pre-copy-dirty-page-tracking=off. + +#. In VM, install iavf driver and vfio-pci driver + + .. code-block:: console + + insmod iavf.ko + modprobe vfio enable_unsafe_noiommu_mode=1 + moodprobe vfio-pci + +#. Bind net device to vfio-pci driver and launch dpdk-testpmd + + .. code-block:: console + + dpdk-testpmd -l 0-1 -- -i + testpmd> set txpkts 64 + testpmd> start tx_first + +#. Start the migration destination Virtual Machine + + .. code-block:: console + + qemu/build/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu host -m 4G -smp 1 -device vfio-pci,host=0000:ca:11.1,x-enable-migration=true,x-pre-copy-dirty-page-tracking=off -drive file=ubuntu-2004.qcow2 -nic user,hostfwd=tcp::5556-:22 -monitor stdio -incoming tcp:127.0.0.1:4444 + + #. Start migration by issuing the command in qemu console + + .. code-block:: console + + migrate -d tcp:127.0.0.1:4444 + +#. Log in the destination VM, and dpdk-testpmd is not interrupt + +.. note:: + + Host kernel version above 5.17 is not supported. Windows Support --------------- diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 4459144140..2ca766b397 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -200,6 +200,9 @@ New Features Enhanced the GRO library to support TCP packets over IPv6 network. +* **Added document for VFIO live migration of Intel E810 device.** + + * Steps to live migrate Intel E810 VFIO devices. Removed Items ------------- -- 2.25.1