From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 833A8AAC9 for ; Mon, 16 Apr 2018 18:36:48 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Apr 2018 09:36:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,459,1517904000"; d="scan'208";a="42347304" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.42]) ([10.237.221.42]) by FMSMGA003.fm.intel.com with ESMTP; 16 Apr 2018 09:36:44 -0700 To: Xiao Wang , anatoly.burakov@intel.com Cc: dev@dpdk.org, maxime.coquelin@redhat.com, zhihong.wang@intel.com, tiwei.bie@intel.com, jianfeng.tan@intel.com, cunming.liang@intel.com, dan.daly@intel.com, thomas@monjalon.net References: <20180415153349.62105-2-xiao.w.wang@intel.com> <20180416153438.79355-1-xiao.w.wang@intel.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= xsFNBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABzSVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+wsF+BBMBAgAoAhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAUCWZR1xgUJB33UawAKCRD5M+tD3xNhH3TID/wNjq1doFXg49WaM7ZXez/1 qwj7U6FQ2eUhlEPX8dXc52cu+iQk8/mssXQtANEx+jndxfvlkikBNgI+mr3m/ho54uQWFZd7 Bv0DVVkLHNkLWK6tT6iISdCgXrQuOv8y5rucEGkJ8dDPsp8Mqr4sBjavRBpczphoa3miKusA HzUEo0SYuHUPmBlbmvKv4PkA5quWtXpkgV2HZ1bW9X0dQkAqEVioAPm5Q6pXJODfV8kaZUtJ z4swEtBnK9XAWm2HccF7KmFh0vv+Zh9lEMnEFt7UPXngY6+xCTo6xV1IVc8EZfDjOip8I4h2 ALMaLgrZwA9VVmHCVOrnO3UZqfGehrwz4O3sUrcmIzxjk3gfBmVRCBfuWGchSpWtZ99U6V1a OEuG+ymyjBDtyymW+KSvmCBl2gIxZHvZFvSRRdDObkkUIskSJ95f/6HBOESRZBOiV9GEAhC5 gI9OAKKF8HQaN/r5KJrkzscjQcjIdV1jXhTkfQ4wH3GJiVM5JxsKsEjjRw7yfSJo2GBEGqMe KMwMLWg4f1DgVtjOuGf10isu+29MvapW2IxKYaHcVc2vHfWbDi2AvBj/VAzKILWbTEgI1VL5 zKpo5p6X2O55oEyeflDiAzrUfvLqB4vmTyqXtW6PdLyZC7kXIzmNu6EBVx9oSgy3CADw5saN 0La9OoCAc7Tn+s7BTQRX1ky+ARAApzQNvXvE2q1LAS+Z+ni2R13Bb1cDS1ZYq1jgpR13+OKN ipzd8MPngRJilXxBaPTErhgzR0vGcNTYhjGMSyFIHVOoBq1VbP1a0Fi/NqWzJOowo/fDfgVy K4vuitc/gCJs+2se4hdZA4EQJxVlNM51lgYDNpjPGIA43MX15OLAip73+ho6NPBMuc5qse3X pAClNhBKfENRCWN428pi3WVkT+ABRTE0taxjJNP7bb+9TQYNRqGwnGzX5/XISv44asWIQCaq vOkXSUJLd//cdVNTqtL1wreCVVR5pMXj7VIrlk07fmmJVALCmGbFr53BMb8O+8dgK2A5mitM n44d+8KdJWOwziRxcaMk/LclmZS3Iv1TERtiWt98Y9AjeAtcgYPkA3ld0BcUKONogP8pHVz1 Ed3s5rDQ91yr1S0wuAzW91fxGUO4wY+uPmxCtFVuBgd9VT9NAKTUL0qHM7CDgCnZPe0TW6Zj 8OqtdCCyAfvU9cW5xWM7Icxhde6AtPxhDSBwE8fL2ZmrDmaA4jmUKXp3i4JxRPSX84S08b+s DWXHPxy10UFU5A7EK/BEbZAKBwn9ROfm+WK+6X5xOGLoRE++OqNuUudxC1GDyLOPaqCbBCS9 +P6HsTHzxsjyJa27n4jcrcuY3P9TEcFJYSZSeSDh8mVGvugi0exnSJrrBZDyVCcAEQEAAcLB ZQQYAQIADwIbDAUCWZR1ZwUJA59cIQAKCRD5M+tD3xNhH5b+D/9XG44Ci6STdcA5RO/ur05J EE3Ux1DCHZ5V7vNAtX/8Wg4l4GZfweauXwuJ1w7Sp7fklwcNC6wsceI+EmNjGMqfIaukGetG +jBGqsQ7moOZodfXUoCK98gblKgt/BPYMVidzlGC8Q/+lZg1+o29sPnwImW+MXt/Z5az/Z17 Qc265g+p5cqJHzq6bpQdnF7Fu6btKU/kv6wJghENvgMXBuyThqsyFReJWFh2wfaKyuix3Zyj ccq7/blkhzIKmtFWgDcgaSc2UAuJU+x9nuYjihW6WobpKP/nlUDu3BIsbIq09UEke+uE/QK+ FJ8PTJkAsXOf1Bc2C0XbW4Y2hf103+YY6L8weUCBsWC5VH5VtVmeuh26ENURclwfeXhWQ9Og 77yzpTXWr5g1Z0oLpYpWPv745J4bE7pv+dzxOrFdM1xNkzY2pvXph/A8OjxZNQklDkHQ7PIB Lki5L2F4XkEOddUUQchJwzMqTPsggPDmGjgLZrqgO+s4ECZK5+nLD3HEpAbPa3JLDaScy+90 Nu1lAqPUHSnP3vYZVw85ZYm6UCxHE4VLMnnJsN09ZhsOSVR+GyP5Nyw9rT1V3lcsuH7M5Naa 2Xobn9m7l9bRCD/Ji8kG15eV1WTxx1HXVQGjdUYDI7UwegBNbwMLh17XDy+3sn/6SgcqtECA Q6pZKA2mTQxEKA== Message-ID: Date: Mon, 16 Apr 2018 17:36:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180416153438.79355-1-xiao.w.wang@intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v8 0/5] add ifcvf vdpa driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Apr 2018 16:36:49 -0000 On 4/16/2018 4:34 PM, Xiao Wang wrote: > IFCVF driver > ============ > The IFCVF vDPA (vhost data path acceleration) driver provides support for the > Intel FPGA 100G VF (IFCVF). IFCVF's datapath is virtio ring compatible, it > works as a HW vhost backend which can send/receive packets to/from virtio > directly by DMA. Besides, it supports dirty page logging and device state > report/restore. This driver enables its vDPA functionality with live migration > feature. > > vDPA mode > ========= > IFCVF's vendor ID and device ID are same as that of virtio net pci device, > with its specific subsystem vendor ID and device ID. To let the device be > probed by IFCVF driver, adding "vdpa=1" parameter helps to specify that this > device is to be used in vDPA mode, rather than polling mode, virtio pmd will > skip when it detects this message. > > Container per device > ==================== > vDPA needs to create different containers for different devices, thus this > patch set adds some APIs in eal/vfio to support multiple container, e.g. > - rte_vfio_create_container > - rte_vfio_destroy_container > - rte_vfio_bind_group > - rte_vfio_unbind_group > > By this extension, a device can be put into a new specific container, rather > than the previous default container. > > IFCVF vDPA details > ================== > Key vDPA driver ops implemented: > - ifcvf_dev_config: > Enable VF data path with virtio information provided by vhost lib, including > IOMMU programming to enable VF DMA to VM's memory, VFIO interrupt setup to > route HW interrupt to virtio driver, create notify relay thread to translate > virtio driver's kick to a MMIO write onto HW, HW queues configuration. > > This function gets called to set up HW data path backend when virtio driver > in VM gets ready. > > - ifcvf_dev_close: > Revoke all the setup in ifcvf_dev_config. > > This function gets called when virtio driver stops device in VM. > > Change log > ========== > v8: > - Rebase on HEAD. > - Move vfio_group definition back to eal_vfio.h. > - Return NULL when vfio group num/fd is not found, let caller handle that. > - Fix wrong API name in commit log. > - Rename bind/unbind function to rte_vfio_container_group_bind/unbind for > consistensy. > - Add note for rte_vfio_container_create and rte_vfio_dma_map and fix typo > in comment. > - Extract out the shared code snip of rte_vfio_dma_map and > rte_vfio_container_dma_map to avoid code duplication. So do for the unmap. > > v7: > - Rebase on HEAD. > - Split the vfio patch into 2 parts, one for data structure extension, one for > adding new API. > - Use static vfio_config array instead of dynamic alloating. > - Change rte_vfio_container_dma_map/unmap's parameters to use (va, iova, len). > > v6: > - Rebase on master branch. > - Document "vdpa" devarg in virtio documentation. > - Rename ifcvf config option to CONFIG_RTE_LIBRTE_IFCVF_VDPA_PMD for > consistensy, and add it into driver documentation. > - Add comments for ifcvf device ID. > - Minor code cleaning. > > v5: > - Fix compilation in BSD, remove the rte_vfio.h including in BSD. > > v4: > - Rebase on Zhihong's latest vDPA lib patch, with vDPA ops names change. > - Remove API "rte_vfio_get_group_fd", "rte_vfio_bind_group" will return the fd. > - Align the vfio_cfg search internal APIs naming. > > v3: > - Add doc and release note for the new driver. > - Remove the vdev concept, make the driver as a PCI driver, it will get probed > by PCI bus driver. > - Rebase on the v4 vDPA lib patch, register a vDPA device instead of a engine. > - Remove the PCI API exposure accordingly. > - Move the MAX_VFIO_CONTAINERS definition to config file. > - Let virtio pmd skips when a virtio device needs to work in vDPA mode. > > v2: > - Rename function pci_get_kernel_driver_by_path to rte_pci_device_kdriver_name > to make the API generic cross Linux and BSD, make it as EXPERIMENTAL. > - Rebase on Zhihong's vDPA v3 patch set. > - Minor code cleanup on vfio extension. > > > Xiao Wang (5): > vfio: extend data structure for multi container > vfio: add multi container support > net/virtio: skip device probe in vdpa mode > net/ifcvf: add ifcvf vdpa driver > doc: add ifcvf driver document and release note Hi Xiao, Getting following build error for 32bit [1], can you please check them? [1] .../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘ifcvf_dma_map’: .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 6 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:22: "size 0x%lx.", i, reg->host_user_addr, ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:37: note: format string is defined here DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 7 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:4: reg->guest_phys_addr, reg->size); ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:48: note: format string is defined here DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 8 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:26: reg->guest_phys_addr, reg->size); ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:14: note: format string is defined here "size 0x%lx.", i, reg->host_user_addr, ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘vdpa_ifcvf_start’: .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:266:39: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].desc = qva_to_gpa(vid, (uint64_t)vq.desc); ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:267:40: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].avail = qva_to_gpa(vid, (uint64_t)vq.avail); ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:268:39: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].used = qva_to_gpa(vid, (uint64_t)vq.used); ^