From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id EE381AAC0 for ; Tue, 17 Apr 2018 09:07:26 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Apr 2018 00:07:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,462,1517904000"; d="scan'208";a="33943230" Received: from dpdk-xiao-1.sh.intel.com ([10.67.110.178]) by orsmga007.jf.intel.com with ESMTP; 17 Apr 2018 00:07:22 -0700 From: Xiao Wang To: ferruh.yigit@intel.com Cc: anatoly.burakov@intel.com, dev@dpdk.org, maxime.coquelin@redhat.com, zhihong.wang@intel.com, tiwei.bie@intel.com, jianfeng.tan@intel.com, cunming.liang@intel.com, dan.daly@intel.com, thomas@monjalon.net, Xiao Wang Date: Tue, 17 Apr 2018 15:06:19 +0800 Message-Id: <20180417070624.18549-1-xiao.w.wang@intel.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180416153438.79355-3-xiao.w.wang@intel.com> References: <20180416153438.79355-3-xiao.w.wang@intel.com> Subject: [dpdk-dev] [PATCH v9 0/5] add ifcvf vdpa driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Apr 2018 07:07:27 -0000 IFCVF driver ============ The IFCVF vDPA (vhost data path acceleration) driver provides support for the Intel FPGA 100G VF (IFCVF). IFCVF's datapath is virtio ring compatible, it works as a HW vhost backend which can send/receive packets to/from virtio directly by DMA. Besides, it supports dirty page logging and device state report/restore. This driver enables its vDPA functionality with live migration feature. vDPA mode ========= IFCVF's vendor ID and device ID are same as that of virtio net pci device, with its specific subsystem vendor ID and device ID. To let the device be probed by IFCVF driver, adding "vdpa=1" parameter helps to specify that this device is to be used in vDPA mode, rather than polling mode, virtio pmd will skip when it detects this message. Container per device ==================== vDPA needs to create different containers for different devices, thus this patch set adds some APIs in eal/vfio to support multiple container, e.g. - rte_vfio_container_create - rte_vfio_container_destroy - rte_vfio_container_group_bind - rte_vfio_container_group_unbind By this extension, a device can be put into a new specific container, rather than the previous default container. Two APIs are added for IOMMU programming for a specified container: - rte_vfio_container_dma_map - rte_vfio_container_dma_unmap IFCVF vDPA details ================== Key vDPA driver ops implemented: - ifcvf_dev_config: Enable VF data path with virtio information provided by vhost lib, including IOMMU programming to enable VF DMA to VM's memory, VFIO interrupt setup to route HW interrupt to virtio driver, create notify relay thread to translate virtio driver's kick to a MMIO write onto HW, HW queues configuration. This function gets called to set up HW data path backend when virtio driver in VM gets ready. - ifcvf_dev_close: Revoke all the setup in ifcvf_dev_config. This function gets called when virtio driver stops device in VM. Change log ========== v9: - Rebase on master tree's HEAD. - Fix compile error on 32-bit platform. v8: - Rebase on HEAD. - Move vfio_group definition back to eal_vfio.h. - Return NULL when vfio group num/fd is not found, let caller handle that. - Fix wrong API name in commit log. - Rename bind/unbind function to rte_vfio_container_group_bind/unbind for consistensy. - Add note for rte_vfio_container_create and rte_vfio_dma_map and fix typo in comment. - Extract out the shared code snip of rte_vfio_dma_map and rte_vfio_container_dma_map to avoid code duplication. So do for the unmap. v7: - Rebase on HEAD. - Split the vfio patch into 2 parts, one for data structure extension, one for adding new API. - Use static vfio_config array instead of dynamic alloating. - Change rte_vfio_container_dma_map/unmap's parameters to use (va, iova, len). v6: - Rebase on master branch. - Document "vdpa" devarg in virtio documentation. - Rename ifcvf config option to CONFIG_RTE_LIBRTE_IFCVF_VDPA_PMD for consistensy, and add it into driver documentation. - Add comments for ifcvf device ID. - Minor code cleaning. v5: - Fix compilation in BSD, remove the rte_vfio.h including in BSD. v4: - Rebase on Zhihong's latest vDPA lib patch, with vDPA ops names change. - Remove API "rte_vfio_get_group_fd", "rte_vfio_bind_group" will return the fd. - Align the vfio_cfg search internal APIs naming. v3: - Add doc and release note for the new driver. - Remove the vdev concept, make the driver as a PCI driver, it will get probed by PCI bus driver. - Rebase on the v4 vDPA lib patch, register a vDPA device instead of a engine. - Remove the PCI API exposure accordingly. - Move the MAX_VFIO_CONTAINERS definition to config file. - Let virtio pmd skips when a virtio device needs to work in vDPA mode. v2: - Rename function pci_get_kernel_driver_by_path to rte_pci_device_kdriver_name to make the API generic cross Linux and BSD, make it as EXPERIMENTAL. - Rebase on Zhihong's vDPA v3 patch set. - Minor code cleanup on vfio extension. Xiao Wang (5): vfio: extend data structure for multi container vfio: add multi container support net/virtio: skip device probe in vdpa mode net/ifcvf: add ifcvf vdpa driver doc: add ifcvf driver document and release note config/common_base | 8 + config/common_linuxapp | 1 + doc/guides/nics/features/ifcvf.ini | 8 + doc/guides/nics/ifcvf.rst | 98 ++++ doc/guides/nics/index.rst | 1 + doc/guides/nics/virtio.rst | 13 + doc/guides/rel_notes/release_18_05.rst | 9 + drivers/net/Makefile | 3 + drivers/net/ifc/Makefile | 35 ++ drivers/net/ifc/base/ifcvf.c | 329 ++++++++++++ drivers/net/ifc/base/ifcvf.h | 160 ++++++ drivers/net/ifc/base/ifcvf_osdep.h | 52 ++ drivers/net/ifc/ifcvf_vdpa.c | 846 +++++++++++++++++++++++++++++++ drivers/net/ifc/rte_ifcvf_version.map | 4 + drivers/net/virtio/virtio_ethdev.c | 43 ++ lib/librte_eal/bsdapp/eal/eal.c | 44 ++ lib/librte_eal/common/include/rte_vfio.h | 128 ++++- lib/librte_eal/linuxapp/eal/eal_vfio.c | 681 +++++++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_vfio.h | 9 +- lib/librte_eal/rte_eal_version.map | 6 + mk/rte.app.mk | 3 + 21 files changed, 2325 insertions(+), 156 deletions(-) create mode 100644 doc/guides/nics/features/ifcvf.ini create mode 100644 doc/guides/nics/ifcvf.rst create mode 100644 drivers/net/ifc/Makefile create mode 100644 drivers/net/ifc/base/ifcvf.c create mode 100644 drivers/net/ifc/base/ifcvf.h create mode 100644 drivers/net/ifc/base/ifcvf_osdep.h create mode 100644 drivers/net/ifc/ifcvf_vdpa.c create mode 100644 drivers/net/ifc/rte_ifcvf_version.map -- 2.15.1