From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <ferruh.yigit@intel.com>
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
by dpdk.org (Postfix) with ESMTP id 833A8AAC9
for <dev@dpdk.org>; Mon, 16 Apr 2018 18:36:48 +0200 (CEST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
16 Apr 2018 09:36:47 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.48,459,1517904000"; d="scan'208";a="42347304"
Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.42])
([10.237.221.42])
by FMSMGA003.fm.intel.com with ESMTP; 16 Apr 2018 09:36:44 -0700
To: Xiao Wang <xiao.w.wang@intel.com>, anatoly.burakov@intel.com
Cc: dev@dpdk.org, maxime.coquelin@redhat.com, zhihong.wang@intel.com,
tiwei.bie@intel.com, jianfeng.tan@intel.com, cunming.liang@intel.com,
dan.daly@intel.com, thomas@monjalon.net
References: <20180415153349.62105-2-xiao.w.wang@intel.com>
<20180416153438.79355-1-xiao.w.wang@intel.com>
From: Ferruh Yigit <ferruh.yigit@intel.com>
Openpgp: preference=signencrypt
Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata=
xsFNBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy
qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ
+iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9
GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb
+dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF
YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy
ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX
CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1
1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz
cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABzSVGZXJydWggWWln
aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+wsF+BBMBAgAoAhsDBgsJCAcDAgYVCAIJCgsE
FgIDAQIeAQIXgAUCWZR1xgUJB33UawAKCRD5M+tD3xNhH3TID/wNjq1doFXg49WaM7ZXez/1
qwj7U6FQ2eUhlEPX8dXc52cu+iQk8/mssXQtANEx+jndxfvlkikBNgI+mr3m/ho54uQWFZd7
Bv0DVVkLHNkLWK6tT6iISdCgXrQuOv8y5rucEGkJ8dDPsp8Mqr4sBjavRBpczphoa3miKusA
HzUEo0SYuHUPmBlbmvKv4PkA5quWtXpkgV2HZ1bW9X0dQkAqEVioAPm5Q6pXJODfV8kaZUtJ
z4swEtBnK9XAWm2HccF7KmFh0vv+Zh9lEMnEFt7UPXngY6+xCTo6xV1IVc8EZfDjOip8I4h2
ALMaLgrZwA9VVmHCVOrnO3UZqfGehrwz4O3sUrcmIzxjk3gfBmVRCBfuWGchSpWtZ99U6V1a
OEuG+ymyjBDtyymW+KSvmCBl2gIxZHvZFvSRRdDObkkUIskSJ95f/6HBOESRZBOiV9GEAhC5
gI9OAKKF8HQaN/r5KJrkzscjQcjIdV1jXhTkfQ4wH3GJiVM5JxsKsEjjRw7yfSJo2GBEGqMe
KMwMLWg4f1DgVtjOuGf10isu+29MvapW2IxKYaHcVc2vHfWbDi2AvBj/VAzKILWbTEgI1VL5
zKpo5p6X2O55oEyeflDiAzrUfvLqB4vmTyqXtW6PdLyZC7kXIzmNu6EBVx9oSgy3CADw5saN
0La9OoCAc7Tn+s7BTQRX1ky+ARAApzQNvXvE2q1LAS+Z+ni2R13Bb1cDS1ZYq1jgpR13+OKN
ipzd8MPngRJilXxBaPTErhgzR0vGcNTYhjGMSyFIHVOoBq1VbP1a0Fi/NqWzJOowo/fDfgVy
K4vuitc/gCJs+2se4hdZA4EQJxVlNM51lgYDNpjPGIA43MX15OLAip73+ho6NPBMuc5qse3X
pAClNhBKfENRCWN428pi3WVkT+ABRTE0taxjJNP7bb+9TQYNRqGwnGzX5/XISv44asWIQCaq
vOkXSUJLd//cdVNTqtL1wreCVVR5pMXj7VIrlk07fmmJVALCmGbFr53BMb8O+8dgK2A5mitM
n44d+8KdJWOwziRxcaMk/LclmZS3Iv1TERtiWt98Y9AjeAtcgYPkA3ld0BcUKONogP8pHVz1
Ed3s5rDQ91yr1S0wuAzW91fxGUO4wY+uPmxCtFVuBgd9VT9NAKTUL0qHM7CDgCnZPe0TW6Zj
8OqtdCCyAfvU9cW5xWM7Icxhde6AtPxhDSBwE8fL2ZmrDmaA4jmUKXp3i4JxRPSX84S08b+s
DWXHPxy10UFU5A7EK/BEbZAKBwn9ROfm+WK+6X5xOGLoRE++OqNuUudxC1GDyLOPaqCbBCS9
+P6HsTHzxsjyJa27n4jcrcuY3P9TEcFJYSZSeSDh8mVGvugi0exnSJrrBZDyVCcAEQEAAcLB
ZQQYAQIADwIbDAUCWZR1ZwUJA59cIQAKCRD5M+tD3xNhH5b+D/9XG44Ci6STdcA5RO/ur05J
EE3Ux1DCHZ5V7vNAtX/8Wg4l4GZfweauXwuJ1w7Sp7fklwcNC6wsceI+EmNjGMqfIaukGetG
+jBGqsQ7moOZodfXUoCK98gblKgt/BPYMVidzlGC8Q/+lZg1+o29sPnwImW+MXt/Z5az/Z17
Qc265g+p5cqJHzq6bpQdnF7Fu6btKU/kv6wJghENvgMXBuyThqsyFReJWFh2wfaKyuix3Zyj
ccq7/blkhzIKmtFWgDcgaSc2UAuJU+x9nuYjihW6WobpKP/nlUDu3BIsbIq09UEke+uE/QK+
FJ8PTJkAsXOf1Bc2C0XbW4Y2hf103+YY6L8weUCBsWC5VH5VtVmeuh26ENURclwfeXhWQ9Og
77yzpTXWr5g1Z0oLpYpWPv745J4bE7pv+dzxOrFdM1xNkzY2pvXph/A8OjxZNQklDkHQ7PIB
Lki5L2F4XkEOddUUQchJwzMqTPsggPDmGjgLZrqgO+s4ECZK5+nLD3HEpAbPa3JLDaScy+90
Nu1lAqPUHSnP3vYZVw85ZYm6UCxHE4VLMnnJsN09ZhsOSVR+GyP5Nyw9rT1V3lcsuH7M5Naa
2Xobn9m7l9bRCD/Ji8kG15eV1WTxx1HXVQGjdUYDI7UwegBNbwMLh17XDy+3sn/6SgcqtECA
Q6pZKA2mTQxEKA==
Message-ID: <c4918ef8-c38f-e363-dc25-077906cf2571@intel.com>
Date: Mon, 16 Apr 2018 17:36:43 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <20180416153438.79355-1-xiao.w.wang@intel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Subject: Re: [dpdk-dev] [PATCH v8 0/5] add ifcvf vdpa driver
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Apr 2018 16:36:49 -0000
On 4/16/2018 4:34 PM, Xiao Wang wrote:
> IFCVF driver
> ============
> The IFCVF vDPA (vhost data path acceleration) driver provides support for the
> Intel FPGA 100G VF (IFCVF). IFCVF's datapath is virtio ring compatible, it
> works as a HW vhost backend which can send/receive packets to/from virtio
> directly by DMA. Besides, it supports dirty page logging and device state
> report/restore. This driver enables its vDPA functionality with live migration
> feature.
>
> vDPA mode
> =========
> IFCVF's vendor ID and device ID are same as that of virtio net pci device,
> with its specific subsystem vendor ID and device ID. To let the device be
> probed by IFCVF driver, adding "vdpa=1" parameter helps to specify that this
> device is to be used in vDPA mode, rather than polling mode, virtio pmd will
> skip when it detects this message.
>
> Container per device
> ====================
> vDPA needs to create different containers for different devices, thus this
> patch set adds some APIs in eal/vfio to support multiple container, e.g.
> - rte_vfio_create_container
> - rte_vfio_destroy_container
> - rte_vfio_bind_group
> - rte_vfio_unbind_group
>
> By this extension, a device can be put into a new specific container, rather
> than the previous default container.
>
> IFCVF vDPA details
> ==================
> Key vDPA driver ops implemented:
> - ifcvf_dev_config:
> Enable VF data path with virtio information provided by vhost lib, including
> IOMMU programming to enable VF DMA to VM's memory, VFIO interrupt setup to
> route HW interrupt to virtio driver, create notify relay thread to translate
> virtio driver's kick to a MMIO write onto HW, HW queues configuration.
>
> This function gets called to set up HW data path backend when virtio driver
> in VM gets ready.
>
> - ifcvf_dev_close:
> Revoke all the setup in ifcvf_dev_config.
>
> This function gets called when virtio driver stops device in VM.
>
> Change log
> ==========
> v8:
> - Rebase on HEAD.
> - Move vfio_group definition back to eal_vfio.h.
> - Return NULL when vfio group num/fd is not found, let caller handle that.
> - Fix wrong API name in commit log.
> - Rename bind/unbind function to rte_vfio_container_group_bind/unbind for
> consistensy.
> - Add note for rte_vfio_container_create and rte_vfio_dma_map and fix typo
> in comment.
> - Extract out the shared code snip of rte_vfio_dma_map and
> rte_vfio_container_dma_map to avoid code duplication. So do for the unmap.
>
> v7:
> - Rebase on HEAD.
> - Split the vfio patch into 2 parts, one for data structure extension, one for
> adding new API.
> - Use static vfio_config array instead of dynamic alloating.
> - Change rte_vfio_container_dma_map/unmap's parameters to use (va, iova, len).
>
> v6:
> - Rebase on master branch.
> - Document "vdpa" devarg in virtio documentation.
> - Rename ifcvf config option to CONFIG_RTE_LIBRTE_IFCVF_VDPA_PMD for
> consistensy, and add it into driver documentation.
> - Add comments for ifcvf device ID.
> - Minor code cleaning.
>
> v5:
> - Fix compilation in BSD, remove the rte_vfio.h including in BSD.
>
> v4:
> - Rebase on Zhihong's latest vDPA lib patch, with vDPA ops names change.
> - Remove API "rte_vfio_get_group_fd", "rte_vfio_bind_group" will return the fd.
> - Align the vfio_cfg search internal APIs naming.
>
> v3:
> - Add doc and release note for the new driver.
> - Remove the vdev concept, make the driver as a PCI driver, it will get probed
> by PCI bus driver.
> - Rebase on the v4 vDPA lib patch, register a vDPA device instead of a engine.
> - Remove the PCI API exposure accordingly.
> - Move the MAX_VFIO_CONTAINERS definition to config file.
> - Let virtio pmd skips when a virtio device needs to work in vDPA mode.
>
> v2:
> - Rename function pci_get_kernel_driver_by_path to rte_pci_device_kdriver_name
> to make the API generic cross Linux and BSD, make it as EXPERIMENTAL.
> - Rebase on Zhihong's vDPA v3 patch set.
> - Minor code cleanup on vfio extension.
>
>
> Xiao Wang (5):
> vfio: extend data structure for multi container
> vfio: add multi container support
> net/virtio: skip device probe in vdpa mode
> net/ifcvf: add ifcvf vdpa driver
> doc: add ifcvf driver document and release note
Hi Xiao,
Getting following build error for 32bit [1], can you please check them?
[1]
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘ifcvf_dma_map’:
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 6 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
"%s(): " fmt "\n", __func__, ##args)
^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:22:
"size 0x%lx.", i, reg->host_user_addr,
~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:37: note: format string is defined here
DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
~~^
%llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 7 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
"%s(): " fmt "\n", __func__, ##args)
^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:4:
reg->guest_phys_addr, reg->size);
~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:48: note: format string is defined here
DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
~~^
%llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 8 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
"%s(): " fmt "\n", __func__, ##args)
^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:26:
reg->guest_phys_addr, reg->size);
~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:14: note: format string is defined here
"size 0x%lx.", i, reg->host_user_addr,
~~^
%llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘vdpa_ifcvf_start’:
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:266:39: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
hw->vring[i].desc = qva_to_gpa(vid, (uint64_t)vq.desc);
^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:267:40: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
hw->vring[i].avail = qva_to_gpa(vid, (uint64_t)vq.avail);
^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:268:39: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
hw->vring[i].used = qva_to_gpa(vid, (uint64_t)vq.used);
^