* [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:15 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type Maxime Coquelin
` (39 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch adds an helper macro to get the rte_vdev_device
pointer from a rte_eth_dev pointer.
This is similar to RTE_ETH_DEV_TO_PCI().
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/bus/vdev/rte_bus_vdev.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/bus/vdev/rte_bus_vdev.h b/drivers/bus/vdev/rte_bus_vdev.h
index d14eeb41b0..f99a41f825 100644
--- a/drivers/bus/vdev/rte_bus_vdev.h
+++ b/drivers/bus/vdev/rte_bus_vdev.h
@@ -34,6 +34,8 @@ struct rte_vdev_device {
#define RTE_DEV_TO_VDEV_CONST(ptr) \
container_of(ptr, const struct rte_vdev_device, device)
+#define RTE_ETH_DEV_TO_VDEV(eth_dev) RTE_DEV_TO_VDEV((eth_dev)->device)
+
static inline const char *
rte_vdev_device_name(const struct rte_vdev_device *dev)
{
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev
2020-12-20 21:13 ` [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev Maxime Coquelin
@ 2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:15 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:13 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev
>
> This patch adds an helper macro to get the rte_vdev_device
> pointer from a rte_eth_dev pointer.
>
> This is similar to RTE_ETH_DEV_TO_PCI().
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/bus/vdev/rte_bus_vdev.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/bus/vdev/rte_bus_vdev.h b/drivers/bus/vdev/rte_bus_vdev.h
> index d14eeb41b0..f99a41f825 100644
> --- a/drivers/bus/vdev/rte_bus_vdev.h
> +++ b/drivers/bus/vdev/rte_bus_vdev.h
> @@ -34,6 +34,8 @@ struct rte_vdev_device {
> #define RTE_DEV_TO_VDEV_CONST(ptr) \
> container_of(ptr, const struct rte_vdev_device, device)
>
> +#define RTE_ETH_DEV_TO_VDEV(eth_dev) RTE_DEV_TO_VDEV((eth_dev)->device)
> +
> static inline const char *
> rte_vdev_device_name(const struct rte_vdev_device *dev)
> {
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev
2020-12-20 21:13 ` [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
@ 2021-01-05 21:15 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:15 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch adds an helper macro to get the rte_vdev_device
> pointer from a rte_eth_dev pointer.
>
> This is similar to RTE_ETH_DEV_TO_PCI().
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
2020-12-20 21:13 ` [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:15 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device Maxime Coquelin
` (38 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch is preliminary work for introducing a bus layer
in Virtio PMD, in order to improve Virtio-user integration.
A new bus type is added to provide a unified way to distinguish
which bus type is used (PCI modern, PCI legacy or Virtio-user).
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 43 ++++++++++++-------------
drivers/net/virtio/virtio_pci.c | 4 +--
drivers/net/virtio/virtio_pci.h | 9 +++++-
drivers/net/virtio/virtio_user_ethdev.c | 2 +-
4 files changed, 32 insertions(+), 26 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 6c233b75ba..b3086297c0 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -592,9 +592,9 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
* we use virtual address. And we need properly set _offset_, please see
* VIRTIO_MBUF_DATA_DMA_ADDR in virtqueue.h for more information.
*/
- if (!hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
vq->offset = offsetof(struct rte_mbuf, buf_iova);
- else {
+ } else if (hw->bus_type == VIRTIO_BUS_USER) {
vq->vq_ring_mem = (uintptr_t)mz->addr;
vq->offset = offsetof(struct rte_mbuf, buf_addr);
if (queue_type == VTNET_TQ)
@@ -746,13 +746,13 @@ virtio_dev_close(struct rte_eth_dev *dev)
virtio_free_queues(hw);
#ifdef RTE_VIRTIO_USER
- if (hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_USER)
virtio_user_dev_uninit(hw->virtio_user_dev);
else
#endif
if (dev->device) {
rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(dev));
- if (!hw->modern)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
rte_pci_ioport_unmap(VTPCI_IO(hw));
}
@@ -1299,7 +1299,7 @@ virtio_intr_unmask(struct rte_eth_dev *dev)
if (rte_intr_ack(dev->intr_handle) < 0)
return -1;
- if (!hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
return 0;
@@ -1313,7 +1313,7 @@ virtio_intr_enable(struct rte_eth_dev *dev)
if (rte_intr_enable(dev->intr_handle) < 0)
return -1;
- if (!hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
return 0;
@@ -1327,7 +1327,7 @@ virtio_intr_disable(struct rte_eth_dev *dev)
if (rte_intr_disable(dev->intr_handle) < 0)
return -1;
- if (!hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
return 0;
@@ -1368,13 +1368,13 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
hw->guest_features);
- if (hw->modern && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
+ if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
PMD_INIT_LOG(ERR,
"VIRTIO_F_VERSION_1 features is not enabled.");
return -1;
}
- if (hw->modern || hw->virtio_user_dev) {
+ if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_USER) {
vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
PMD_INIT_LOG(ERR,
@@ -1709,7 +1709,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
- if (!hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
/* If host does not support both status and MSI-X then disable LSC */
@@ -1856,7 +1856,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
static int
virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
{
- if (hw->modern) {
+ if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
/*
* We don't have to re-parse the PCI config space, since
* rte_pci_map_device() makes sure the mapped address
@@ -1872,7 +1872,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
PMD_INIT_LOG(DEBUG, "failed to map pci device!");
return -1;
}
- } else {
+ } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
return -1;
}
@@ -1883,15 +1883,14 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
static void
virtio_set_vtpci_ops(struct virtio_hw *hw)
{
-#ifdef RTE_VIRTIO_USER
- if (hw->virtio_user_dev)
+ if (hw->bus_type == VIRTIO_BUS_USER)
VTPCI_OPS(hw) = &virtio_user_ops;
- else
-#endif
- if (hw->modern)
+ else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
VTPCI_OPS(hw) = &modern_ops;
- else
+ else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
VTPCI_OPS(hw) = &legacy_ops;
+
+ return;
}
/*
@@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
- if (!hw->virtio_user_dev) {
+ if (hw->bus_type != VIRTIO_BUS_USER) {
ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
if (ret)
return ret;
@@ -1950,7 +1949,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
/* For virtio_user case the hw->virtio_user_dev is populated by
* virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
*/
- if (!hw->virtio_user_dev) {
+ if (hw->bus_type != VIRTIO_BUS_USER) {
ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
if (ret)
goto err_vtpci_init;
@@ -1982,9 +1981,9 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
return 0;
err_virtio_init:
- if (!hw->virtio_user_dev) {
+ if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
- if (!hw->modern)
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
rte_pci_ioport_unmap(VTPCI_IO(hw));
}
err_vtpci_init:
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index d6b950ee69..1692268f30 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -697,7 +697,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
if (virtio_read_caps(dev, hw) == 0) {
PMD_INIT_LOG(INFO, "modern virtio pci detected.");
virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
- hw->modern = 1;
+ hw->bus_type = VIRTIO_BUS_PCI_MODERN;
return 0;
}
@@ -716,7 +716,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
}
virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
- hw->modern = 0;
+ hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
return 0;
}
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index ab61e911b8..6388f0a74d 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -243,7 +243,15 @@ struct virtio_pci_ops {
struct virtio_net_config;
+enum virtio_bus_type {
+ VIRTIO_BUS_UNKNOWN,
+ VIRTIO_BUS_PCI_LEGACY,
+ VIRTIO_BUS_PCI_MODERN,
+ VIRTIO_BUS_USER,
+};
+
struct virtio_hw {
+ enum virtio_bus_type bus_type;
struct virtnet_ctl *cvq;
uint64_t req_guest_features;
uint64_t guest_features;
@@ -253,7 +261,6 @@ struct virtio_hw {
uint16_t vtnet_hdr_size;
uint8_t vlan_strip;
uint8_t use_msix;
- uint8_t modern;
uint8_t use_vec_rx;
uint8_t use_vec_tx;
uint8_t use_inorder_rx;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 40345193e6..516d0ee577 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -629,7 +629,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
* Here just pretend that we support msix.
*/
hw->use_msix = 1;
- hw->modern = 0;
+ hw->bus_type = VIRTIO_BUS_USER;
hw->use_vec_rx = 0;
hw->use_vec_tx = 0;
hw->use_inorder_rx = 0;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2020-12-20 21:13 ` [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type Maxime Coquelin
@ 2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:15 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:13 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 02/40] net/virtio: Introduce Virtio bus type
>
> This patch is preliminary work for introducing a bus layer
> in Virtio PMD, in order to improve Virtio-user integration.
>
> A new bus type is added to provide a unified way to distinguish
> which bus type is used (PCI modern, PCI legacy or Virtio-user).
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 43 ++++++++++++-------------
> drivers/net/virtio/virtio_pci.c | 4 +--
> drivers/net/virtio/virtio_pci.h | 9 +++++-
> drivers/net/virtio/virtio_user_ethdev.c | 2 +-
> 4 files changed, 32 insertions(+), 26 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 6c233b75ba..b3086297c0 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -592,9 +592,9 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> * we use virtual address. And we need properly set _offset_, please see
> * VIRTIO_MBUF_DATA_DMA_ADDR in virtqueue.h for more information.
> */
> - if (!hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN) {
> vq->offset = offsetof(struct rte_mbuf, buf_iova);
> - else {
> + } else if (hw->bus_type == VIRTIO_BUS_USER) {
> vq->vq_ring_mem = (uintptr_t)mz->addr;
> vq->offset = offsetof(struct rte_mbuf, buf_addr);
> if (queue_type == VTNET_TQ)
> @@ -746,13 +746,13 @@ virtio_dev_close(struct rte_eth_dev *dev)
> virtio_free_queues(hw);
>
> #ifdef RTE_VIRTIO_USER
> - if (hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_USER)
> virtio_user_dev_uninit(hw->virtio_user_dev);
> else
> #endif
> if (dev->device) {
> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(dev));
> - if (!hw->modern)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> rte_pci_ioport_unmap(VTPCI_IO(hw));
> }
>
> @@ -1299,7 +1299,7 @@ virtio_intr_unmask(struct rte_eth_dev *dev)
> if (rte_intr_ack(dev->intr_handle) < 0)
> return -1;
>
> - if (!hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
>
> return 0;
> @@ -1313,7 +1313,7 @@ virtio_intr_enable(struct rte_eth_dev *dev)
> if (rte_intr_enable(dev->intr_handle) < 0)
> return -1;
>
> - if (!hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
>
> return 0;
> @@ -1327,7 +1327,7 @@ virtio_intr_disable(struct rte_eth_dev *dev)
> if (rte_intr_disable(dev->intr_handle) < 0)
> return -1;
>
> - if (!hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
>
> return 0;
> @@ -1368,13 +1368,13 @@ virtio_negotiate_features(struct virtio_hw *hw,
> uint64_t req_features)
> PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
> hw->guest_features);
>
> - if (hw->modern && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw,
> VIRTIO_F_VERSION_1)) {
> PMD_INIT_LOG(ERR,
> "VIRTIO_F_VERSION_1 features is not enabled.");
> return -1;
> }
>
> - if (hw->modern || hw->virtio_user_dev) {
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type ==
> VIRTIO_BUS_USER) {
> vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
> if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
> PMD_INIT_LOG(ERR,
> @@ -1709,7 +1709,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
>
> hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
>
> - if (!hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
>
> /* If host does not support both status and MSI-X then disable LSC */
> @@ -1856,7 +1856,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> static int
> virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> {
> - if (hw->modern) {
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
> /*
> * We don't have to re-parse the PCI config space, since
> * rte_pci_map_device() makes sure the mapped address
> @@ -1872,7 +1872,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct
> virtio_hw *hw)
> PMD_INIT_LOG(DEBUG, "failed to map pci device!");
> return -1;
> }
> - } else {
> + } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
> if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
> return -1;
> }
> @@ -1883,15 +1883,14 @@ virtio_remap_pci(struct rte_pci_device *pci_dev,
> struct virtio_hw *hw)
> static void
> virtio_set_vtpci_ops(struct virtio_hw *hw)
> {
> -#ifdef RTE_VIRTIO_USER
> - if (hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_USER)
> VTPCI_OPS(hw) = &virtio_user_ops;
> - else
> -#endif
> - if (hw->modern)
> + else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
> VTPCI_OPS(hw) = &modern_ops;
> - else
> + else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> VTPCI_OPS(hw) = &legacy_ops;
> +
> + return;
> }
>
> /*
> @@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type != VIRTIO_BUS_USER) {
> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> if (ret)
> return ret;
> @@ -1950,7 +1949,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> /* For virtio_user case the hw->virtio_user_dev is populated by
> * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
> */
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type != VIRTIO_BUS_USER) {
> ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> if (ret)
> goto err_vtpci_init;
> @@ -1982,9 +1981,9 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> return 0;
>
> err_virtio_init:
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type ==
> VIRTIO_BUS_PCI_LEGACY) {
> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> - if (!hw->modern)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> rte_pci_ioport_unmap(VTPCI_IO(hw));
> }
> err_vtpci_init:
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index d6b950ee69..1692268f30 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -697,7 +697,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw
> *hw)
> if (virtio_read_caps(dev, hw) == 0) {
> PMD_INIT_LOG(INFO, "modern virtio pci detected.");
> virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
> - hw->modern = 1;
> + hw->bus_type = VIRTIO_BUS_PCI_MODERN;
> return 0;
> }
>
> @@ -716,7 +716,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw
> *hw)
> }
>
> virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
> - hw->modern = 0;
> + hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
>
> return 0;
> }
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index ab61e911b8..6388f0a74d 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -243,7 +243,15 @@ struct virtio_pci_ops {
>
> struct virtio_net_config;
>
> +enum virtio_bus_type {
> + VIRTIO_BUS_UNKNOWN,
> + VIRTIO_BUS_PCI_LEGACY,
> + VIRTIO_BUS_PCI_MODERN,
> + VIRTIO_BUS_USER,
> +};
> +
> struct virtio_hw {
> + enum virtio_bus_type bus_type;
> struct virtnet_ctl *cvq;
> uint64_t req_guest_features;
> uint64_t guest_features;
> @@ -253,7 +261,6 @@ struct virtio_hw {
> uint16_t vtnet_hdr_size;
> uint8_t vlan_strip;
> uint8_t use_msix;
> - uint8_t modern;
> uint8_t use_vec_rx;
> uint8_t use_vec_tx;
> uint8_t use_inorder_rx;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 40345193e6..516d0ee577 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -629,7 +629,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
> * Here just pretend that we support msix.
> */
> hw->use_msix = 1;
> - hw->modern = 0;
> + hw->bus_type = VIRTIO_BUS_USER;
> hw->use_vec_rx = 0;
> hw->use_vec_tx = 0;
> hw->use_inorder_rx = 0;
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2020-12-20 21:13 ` [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
@ 2021-01-05 21:15 ` David Marchand
2021-01-14 9:24 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:15 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch is preliminary work for introducing a bus layer
> in Virtio PMD, in order to improve Virtio-user integration.
>
> A new bus type is added to provide a unified way to distinguish
> which bus type is used (PCI modern, PCI legacy or Virtio-user).
In dpdk, we don't use a capital letter for starting a title.
> @@ -1883,15 +1883,14 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> static void
> virtio_set_vtpci_ops(struct virtio_hw *hw)
> {
> -#ifdef RTE_VIRTIO_USER
Too soon to remove this check, since virtio_user_ops comes from
virtio_user_ethdev.c.
This will break compilation on FreeBSD.
> - if (hw->virtio_user_dev)
> + if (hw->bus_type == VIRTIO_BUS_USER)
> VTPCI_OPS(hw) = &virtio_user_ops;
> - else
> -#endif
> - if (hw->modern)
> + else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
> VTPCI_OPS(hw) = &modern_ops;
> - else
> + else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> VTPCI_OPS(hw) = &legacy_ops;
> +
> + return;
> }
>
> /*
> @@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type != VIRTIO_BUS_USER) {
In the rest of the patch, we check for PCI types when dealing with PCI
code, so I'd rather be consistent and check for modern and legacy pci
types here too.
> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> if (ret)
> return ret;
> @@ -1950,7 +1949,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> /* For virtio_user case the hw->virtio_user_dev is populated by
> * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
> */
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type != VIRTIO_BUS_USER) {
Idem.
> ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> if (ret)
> goto err_vtpci_init;
> @@ -1982,9 +1981,9 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> return 0;
>
> err_virtio_init:
> - if (!hw->virtio_user_dev) {
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> - if (!hw->modern)
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> rte_pci_ioport_unmap(VTPCI_IO(hw));
> }
> err_vtpci_init:
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2021-01-05 21:15 ` David Marchand
@ 2021-01-14 9:24 ` Maxime Coquelin
2021-01-14 10:54 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 9:24 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/5/21 10:15 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch is preliminary work for introducing a bus layer
>> in Virtio PMD, in order to improve Virtio-user integration.
>>
>> A new bus type is added to provide a unified way to distinguish
>> which bus type is used (PCI modern, PCI legacy or Virtio-user).
>
> In dpdk, we don't use a capital letter for starting a title.
>
>
>> @@ -1883,15 +1883,14 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
>> static void
>> virtio_set_vtpci_ops(struct virtio_hw *hw)
>> {
>> -#ifdef RTE_VIRTIO_USER
>
> Too soon to remove this check, since virtio_user_ops comes from
> virtio_user_ethdev.c.
> This will break compilation on FreeBSD.
Agree, I restored it back in v2.
>
>> - if (hw->virtio_user_dev)
>> + if (hw->bus_type == VIRTIO_BUS_USER)
>> VTPCI_OPS(hw) = &virtio_user_ops;
>> - else
>> -#endif
>> - if (hw->modern)
>> + else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
>> VTPCI_OPS(hw) = &modern_ops;
>> - else
>> + else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
>> VTPCI_OPS(hw) = &legacy_ops;
>> +
>> + return;
>> }
>>
>> /*
>> @@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>>
>> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
>> - if (!hw->virtio_user_dev) {
>> + if (hw->bus_type != VIRTIO_BUS_USER) {
>
> In the rest of the patch, we check for PCI types when dealing with PCI
> code, so I'd rather be consistent and check for modern and legacy pci
> types here too.
Agree, it is not consistent. I fixed it here and below for v2.
>
>> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
>> if (ret)
>> return ret;
>> @@ -1950,7 +1949,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>> /* For virtio_user case the hw->virtio_user_dev is populated by
>> * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
>> */
>> - if (!hw->virtio_user_dev) {
>> + if (hw->bus_type != VIRTIO_BUS_USER) {
>
> Idem.
>
>
>> ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
>> if (ret)
>> goto err_vtpci_init;
>> @@ -1982,9 +1981,9 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>> return 0;
>>
>> err_virtio_init:
>> - if (!hw->virtio_user_dev) {
>> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
>> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
>> - if (!hw->modern)
>> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
>> rte_pci_ioport_unmap(VTPCI_IO(hw));
>> }
>> err_vtpci_init:
>
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2021-01-14 9:24 ` Maxime Coquelin
@ 2021-01-14 10:54 ` Maxime Coquelin
2021-01-14 11:55 ` David Marchand
0 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 10:54 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/14/21 10:24 AM, Maxime Coquelin wrote:
>
>
> On 1/5/21 10:15 PM, David Marchand wrote:
>> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
>> <maxime.coquelin@redhat.com> wrote:
>>>
>>> This patch is preliminary work for introducing a bus layer
>>> in Virtio PMD, in order to improve Virtio-user integration.
>>>
>>> A new bus type is added to provide a unified way to distinguish
>>> which bus type is used (PCI modern, PCI legacy or Virtio-user).
>>
>> In dpdk, we don't use a capital letter for starting a title.
>>
>>
>>> @@ -1883,15 +1883,14 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
>>> static void
>>> virtio_set_vtpci_ops(struct virtio_hw *hw)
>>> {
>>> -#ifdef RTE_VIRTIO_USER
>>
>> Too soon to remove this check, since virtio_user_ops comes from
>> virtio_user_ethdev.c.
>> This will break compilation on FreeBSD.
>
> Agree, I restored it back in v2.
>
>>
>>> - if (hw->virtio_user_dev)
>>> + if (hw->bus_type == VIRTIO_BUS_USER)
>>> VTPCI_OPS(hw) = &virtio_user_ops;
>>> - else
>>> -#endif
>>> - if (hw->modern)
>>> + else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
>>> VTPCI_OPS(hw) = &modern_ops;
>>> - else
>>> + else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
>>> VTPCI_OPS(hw) = &legacy_ops;
>>> +
>>> + return;
>>> }
>>>
>>> /*
>>> @@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>>> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>>>
>>> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
>>> - if (!hw->virtio_user_dev) {
>>> + if (hw->bus_type != VIRTIO_BUS_USER) {
>>
>> In the rest of the patch, we check for PCI types when dealing with PCI
>> code, so I'd rather be consistent and check for modern and legacy pci
>> types here too.
>
> Agree, it is not consistent. I fixed it here and below for v2.
Wait, as we discussed off-list, we need hw->bus_type != VIRTIO_BUS_USER,
as the bus_type is still unknown for PCI bus at that stage.
>>
>>> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
>>> if (ret)
>>> return ret;
>>> @@ -1950,7 +1949,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>>> /* For virtio_user case the hw->virtio_user_dev is populated by
>>> * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
>>> */
>>> - if (!hw->virtio_user_dev) {
>>> + if (hw->bus_type != VIRTIO_BUS_USER) {
>>
>> Idem.
Same here.
>>
>>
>>> ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
bus_type is set to VIRTIO_BUS_PCI_MODERN or VIRTIO_BUS_PCI_LEGACY in
vtcpi_init().
>>> if (ret)
>>> goto err_vtpci_init;
>>> @@ -1982,9 +1981,9 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>>> return 0;
>>>
>>> err_virtio_init:
>>> - if (!hw->virtio_user_dev) {
>>> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
>>> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
>>> - if (!hw->modern)
>>> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
>>> rte_pci_ioport_unmap(VTPCI_IO(hw));
>>> }
>>> err_vtpci_init:
>>
>>
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type
2021-01-14 10:54 ` Maxime Coquelin
@ 2021-01-14 11:55 ` David Marchand
0 siblings, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-14 11:55 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Thu, Jan 14, 2021 at 11:55 AM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> >>> @@ -1919,7 +1918,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> >>> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
> >>>
> >>> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> >>> - if (!hw->virtio_user_dev) {
> >>> + if (hw->bus_type != VIRTIO_BUS_USER) {
> >>
> >> In the rest of the patch, we check for PCI types when dealing with PCI
> >> code, so I'd rather be consistent and check for modern and legacy pci
> >> types here too.
> >
> > Agree, it is not consistent. I fixed it here and below for v2.
>
> Wait, as we discussed off-list, we need hw->bus_type != VIRTIO_BUS_USER,
> as the bus_type is still unknown for PCI bus at that stage.
Yep, thanks for double checking.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
2020-12-20 21:13 ` [dpdk-dev] [PATCH 01/40] bus/vdev: add helper to get vdev from eth dev Maxime Coquelin
2020-12-20 21:13 ` [dpdk-dev] [PATCH 02/40] net/virtio: Introduce Virtio bus type Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:16 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata Maxime Coquelin
` (37 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves the virtio_hw structure into the virtio_user_dev
structure, with the goal of making virtio_hw bus-agnostic.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 2 +-
drivers/net/virtio/virtio_pci.h | 1 -
.../net/virtio/virtio_user/virtio_user_dev.h | 1 +
drivers/net/virtio/virtio_user_ethdev.c | 62 ++++++++-----------
4 files changed, 27 insertions(+), 39 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index b3086297c0..3ace25ac80 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -747,7 +747,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
#ifdef RTE_VIRTIO_USER
if (hw->bus_type == VIRTIO_BUS_USER)
- virtio_user_dev_uninit(hw->virtio_user_dev);
+ virtio_user_dev_uninit(dev->data->dev_private);
else
#endif
if (dev->device) {
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 6388f0a74d..b35a596169 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -277,7 +277,6 @@ struct virtio_hw {
uint16_t *notify_base;
struct virtio_pci_common_cfg *common_cfg;
struct virtio_net_config *dev_cfg;
- void *virtio_user_dev;
/*
* App management thread and virtio interrupt handler thread
* both can change device state, this lock is meant to avoid
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index e053897d8f..59f4dd1f24 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -24,6 +24,7 @@ struct virtio_user_queue {
};
struct virtio_user_dev {
+ struct virtio_hw hw;
enum virtio_user_backend_type backend_type;
/* for vhost_user backend */
int vhostfd;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 516d0ee577..1f1f63a1a5 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -26,13 +26,13 @@
#include "virtio_user/virtio_user_dev.h"
#include "virtio_user/vhost.h"
-#define virtio_user_get_dev(hw) \
- ((struct virtio_user_dev *)(hw)->virtio_user_dev)
+#define virtio_user_get_dev(hw) container_of(hw, struct virtio_user_dev, hw)
static void
-virtio_user_reset_queues_packed(struct rte_eth_dev *dev)
+virtio_user_reset_queues_packed(struct rte_eth_dev *eth_dev)
{
- struct virtio_hw *hw = dev->data->dev_private;
+ struct virtio_user_dev *dev = eth_dev->data->dev_private;
+ struct virtio_hw *hw = &dev->hw;
struct virtnet_rx *rxvq;
struct virtnet_tx *txvq;
uint16_t i;
@@ -48,14 +48,14 @@ virtio_user_reset_queues_packed(struct rte_eth_dev *dev)
rte_delay_ms(1);
/* Vring reset for each Tx queue and Rx queue. */
- for (i = 0; i < dev->data->nb_rx_queues; i++) {
- rxvq = dev->data->rx_queues[i];
+ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+ rxvq = eth_dev->data->rx_queues[i];
virtqueue_rxvq_reset_packed(rxvq->vq);
- virtio_dev_rx_queue_setup_finish(dev, i);
+ virtio_dev_rx_queue_setup_finish(eth_dev, i);
}
- for (i = 0; i < dev->data->nb_tx_queues; i++) {
- txvq = dev->data->tx_queues[i];
+ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+ txvq = eth_dev->data->tx_queues[i];
virtqueue_txvq_reset_packed(txvq->vq);
}
@@ -69,7 +69,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
{
int ret, connectfd, old_status;
struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
- struct virtio_hw *hw = eth_dev->data->dev_private;
+ struct virtio_hw *hw = &dev->hw;
uint64_t protocol_features;
connectfd = accept(dev->listenfd, NULL, NULL);
@@ -605,21 +605,15 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
struct virtio_hw *hw;
struct virtio_user_dev *dev;
- eth_dev = rte_eth_vdev_allocate(vdev, sizeof(*hw));
+ eth_dev = rte_eth_vdev_allocate(vdev, sizeof(*dev));
if (!eth_dev) {
PMD_INIT_LOG(ERR, "cannot alloc rte_eth_dev");
return NULL;
}
data = eth_dev->data;
- hw = eth_dev->data->dev_private;
-
- dev = rte_zmalloc(NULL, sizeof(*dev), 0);
- if (!dev) {
- PMD_INIT_LOG(ERR, "malloc virtio_user_dev failed");
- rte_eth_dev_release_port(eth_dev);
- return NULL;
- }
+ dev = eth_dev->data->dev_private;
+ hw = &dev->hw;
hw->port_id = data->port_id;
dev->port_id = data->port_id;
@@ -634,17 +628,13 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
hw->use_vec_tx = 0;
hw->use_inorder_rx = 0;
hw->use_inorder_tx = 0;
- hw->virtio_user_dev = dev;
+
return eth_dev;
}
static void
virtio_user_eth_dev_free(struct rte_eth_dev *eth_dev)
{
- struct rte_eth_dev_data *data = eth_dev->data;
- struct virtio_hw *hw = data->dev_private;
-
- rte_free(hw->virtio_user_dev);
rte_eth_dev_release_port(eth_dev);
}
@@ -653,11 +643,12 @@ virtio_user_eth_dev_free(struct rte_eth_dev *eth_dev)
* Returns 0 on success.
*/
static int
-virtio_user_pmd_probe(struct rte_vdev_device *dev)
+virtio_user_pmd_probe(struct rte_vdev_device *vdev)
{
struct rte_kvargs *kvlist = NULL;
struct rte_eth_dev *eth_dev;
struct virtio_hw *hw;
+ struct virtio_user_dev *dev;
enum virtio_user_backend_type backend_type = VIRTIO_USER_BACKEND_UNKNOWN;
uint64_t queues = VIRTIO_USER_DEF_Q_NUM;
uint64_t cq = VIRTIO_USER_DEF_CQ_EN;
@@ -673,7 +664,7 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
int ret = -1;
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
- const char *name = rte_vdev_device_name(dev);
+ const char *name = rte_vdev_device_name(vdev);
eth_dev = rte_eth_dev_attach_secondary(name);
if (!eth_dev) {
PMD_INIT_LOG(ERR, "Failed to probe %s", name);
@@ -687,12 +678,12 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
}
eth_dev->dev_ops = &virtio_user_secondary_eth_dev_ops;
- eth_dev->device = &dev->device;
+ eth_dev->device = &vdev->device;
rte_eth_dev_probing_finish(eth_dev);
return 0;
}
- kvlist = rte_kvargs_parse(rte_vdev_device_args(dev), valid_args);
+ kvlist = rte_kvargs_parse(rte_vdev_device_args(vdev), valid_args);
if (!kvlist) {
PMD_INIT_LOG(ERR, "error when parsing param");
goto end;
@@ -832,14 +823,15 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
}
}
- eth_dev = virtio_user_eth_dev_alloc(dev);
+ eth_dev = virtio_user_eth_dev_alloc(vdev);
if (!eth_dev) {
PMD_INIT_LOG(ERR, "virtio_user fails to alloc device");
goto end;
}
- hw = eth_dev->data->dev_private;
- if (virtio_user_dev_init(hw->virtio_user_dev, path, queues, cq,
+ dev = eth_dev->data->dev_private;
+ hw = &dev->hw;
+ if (virtio_user_dev_init(dev, path, queues, cq,
queue_size, mac_addr, &ifname, server_mode,
mrg_rxbuf, in_order, packed_vq, backend_type) < 0) {
PMD_INIT_LOG(ERR, "virtio_user_dev_init fails");
@@ -912,7 +904,6 @@ static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
const char *name;
struct rte_eth_dev *eth_dev;
struct virtio_user_dev *dev;
- struct virtio_hw *hw;
if (!vdev)
return -EINVAL;
@@ -923,8 +914,7 @@ static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
if (!eth_dev)
return 0;
- hw = (struct virtio_hw *)eth_dev->data->dev_private;
- dev = hw->virtio_user_dev;
+ dev = eth_dev->data->dev_private;
if (dev->ops->dma_map)
return dev->ops->dma_map(dev, addr, iova, len);
@@ -938,7 +928,6 @@ static int virtio_user_pmd_dma_unmap(struct rte_vdev_device *vdev, void *addr,
const char *name;
struct rte_eth_dev *eth_dev;
struct virtio_user_dev *dev;
- struct virtio_hw *hw;
if (!vdev)
return -EINVAL;
@@ -949,8 +938,7 @@ static int virtio_user_pmd_dma_unmap(struct rte_vdev_device *vdev, void *addr,
if (!eth_dev)
return 0;
- hw = (struct virtio_hw *)eth_dev->data->dev_private;
- dev = hw->virtio_user_dev;
+ dev = eth_dev->data->dev_private;
if (dev->ops->dma_unmap)
return dev->ops->dma_unmap(dev, addr, iova, len);
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device
2020-12-20 21:13 ` [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device Maxime Coquelin
@ 2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:16 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:13 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 03/40] net/virtio: refactor virtio-user device
>
> This patch moves the virtio_hw structure into the virtio_user_dev
> structure, with the goal of making virtio_hw bus-agnostic.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 2 +-
> drivers/net/virtio/virtio_pci.h | 1 -
> .../net/virtio/virtio_user/virtio_user_dev.h | 1 +
> drivers/net/virtio/virtio_user_ethdev.c | 62 ++++++++-----------
> 4 files changed, 27 insertions(+), 39 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index b3086297c0..3ace25ac80 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -747,7 +747,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
>
> #ifdef RTE_VIRTIO_USER
> if (hw->bus_type == VIRTIO_BUS_USER)
> - virtio_user_dev_uninit(hw->virtio_user_dev);
> + virtio_user_dev_uninit(dev->data->dev_private);
> else
> #endif
> if (dev->device) {
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 6388f0a74d..b35a596169 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -277,7 +277,6 @@ struct virtio_hw {
> uint16_t *notify_base;
> struct virtio_pci_common_cfg *common_cfg;
> struct virtio_net_config *dev_cfg;
> - void *virtio_user_dev;
> /*
> * App management thread and virtio interrupt handler thread
> * both can change device state, this lock is meant to avoid
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index e053897d8f..59f4dd1f24 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -24,6 +24,7 @@ struct virtio_user_queue {
> };
>
> struct virtio_user_dev {
> + struct virtio_hw hw;
> enum virtio_user_backend_type backend_type;
> /* for vhost_user backend */
> int vhostfd;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 516d0ee577..1f1f63a1a5 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -26,13 +26,13 @@
> #include "virtio_user/virtio_user_dev.h"
> #include "virtio_user/vhost.h"
>
> -#define virtio_user_get_dev(hw) \
> - ((struct virtio_user_dev *)(hw)->virtio_user_dev)
> +#define virtio_user_get_dev(hw) container_of(hw, struct virtio_user_dev, hw)
>
> static void
> -virtio_user_reset_queues_packed(struct rte_eth_dev *dev)
> +virtio_user_reset_queues_packed(struct rte_eth_dev *eth_dev)
> {
> - struct virtio_hw *hw = dev->data->dev_private;
> + struct virtio_user_dev *dev = eth_dev->data->dev_private;
> + struct virtio_hw *hw = &dev->hw;
> struct virtnet_rx *rxvq;
> struct virtnet_tx *txvq;
> uint16_t i;
> @@ -48,14 +48,14 @@ virtio_user_reset_queues_packed(struct rte_eth_dev *dev)
> rte_delay_ms(1);
>
> /* Vring reset for each Tx queue and Rx queue. */
> - for (i = 0; i < dev->data->nb_rx_queues; i++) {
> - rxvq = dev->data->rx_queues[i];
> + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
> + rxvq = eth_dev->data->rx_queues[i];
> virtqueue_rxvq_reset_packed(rxvq->vq);
> - virtio_dev_rx_queue_setup_finish(dev, i);
> + virtio_dev_rx_queue_setup_finish(eth_dev, i);
> }
>
> - for (i = 0; i < dev->data->nb_tx_queues; i++) {
> - txvq = dev->data->tx_queues[i];
> + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
> + txvq = eth_dev->data->tx_queues[i];
> virtqueue_txvq_reset_packed(txvq->vq);
> }
>
> @@ -69,7 +69,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
> {
> int ret, connectfd, old_status;
> struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
> - struct virtio_hw *hw = eth_dev->data->dev_private;
> + struct virtio_hw *hw = &dev->hw;
> uint64_t protocol_features;
>
> connectfd = accept(dev->listenfd, NULL, NULL);
> @@ -605,21 +605,15 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
> struct virtio_hw *hw;
> struct virtio_user_dev *dev;
>
> - eth_dev = rte_eth_vdev_allocate(vdev, sizeof(*hw));
> + eth_dev = rte_eth_vdev_allocate(vdev, sizeof(*dev));
> if (!eth_dev) {
> PMD_INIT_LOG(ERR, "cannot alloc rte_eth_dev");
> return NULL;
> }
>
> data = eth_dev->data;
> - hw = eth_dev->data->dev_private;
> -
> - dev = rte_zmalloc(NULL, sizeof(*dev), 0);
> - if (!dev) {
> - PMD_INIT_LOG(ERR, "malloc virtio_user_dev failed");
> - rte_eth_dev_release_port(eth_dev);
> - return NULL;
> - }
> + dev = eth_dev->data->dev_private;
> + hw = &dev->hw;
>
> hw->port_id = data->port_id;
> dev->port_id = data->port_id;
> @@ -634,17 +628,13 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
> hw->use_vec_tx = 0;
> hw->use_inorder_rx = 0;
> hw->use_inorder_tx = 0;
> - hw->virtio_user_dev = dev;
> +
> return eth_dev;
> }
>
> static void
> virtio_user_eth_dev_free(struct rte_eth_dev *eth_dev)
> {
> - struct rte_eth_dev_data *data = eth_dev->data;
> - struct virtio_hw *hw = data->dev_private;
> -
> - rte_free(hw->virtio_user_dev);
> rte_eth_dev_release_port(eth_dev);
> }
>
> @@ -653,11 +643,12 @@ virtio_user_eth_dev_free(struct rte_eth_dev *eth_dev)
> * Returns 0 on success.
> */
> static int
> -virtio_user_pmd_probe(struct rte_vdev_device *dev)
> +virtio_user_pmd_probe(struct rte_vdev_device *vdev)
> {
> struct rte_kvargs *kvlist = NULL;
> struct rte_eth_dev *eth_dev;
> struct virtio_hw *hw;
> + struct virtio_user_dev *dev;
> enum virtio_user_backend_type backend_type = VIRTIO_USER_BACKEND_UNKNOWN;
> uint64_t queues = VIRTIO_USER_DEF_Q_NUM;
> uint64_t cq = VIRTIO_USER_DEF_CQ_EN;
> @@ -673,7 +664,7 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
> int ret = -1;
>
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> - const char *name = rte_vdev_device_name(dev);
> + const char *name = rte_vdev_device_name(vdev);
> eth_dev = rte_eth_dev_attach_secondary(name);
> if (!eth_dev) {
> PMD_INIT_LOG(ERR, "Failed to probe %s", name);
> @@ -687,12 +678,12 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
> }
>
> eth_dev->dev_ops = &virtio_user_secondary_eth_dev_ops;
> - eth_dev->device = &dev->device;
> + eth_dev->device = &vdev->device;
> rte_eth_dev_probing_finish(eth_dev);
> return 0;
> }
>
> - kvlist = rte_kvargs_parse(rte_vdev_device_args(dev), valid_args);
> + kvlist = rte_kvargs_parse(rte_vdev_device_args(vdev), valid_args);
> if (!kvlist) {
> PMD_INIT_LOG(ERR, "error when parsing param");
> goto end;
> @@ -832,14 +823,15 @@ virtio_user_pmd_probe(struct rte_vdev_device *dev)
> }
> }
>
> - eth_dev = virtio_user_eth_dev_alloc(dev);
> + eth_dev = virtio_user_eth_dev_alloc(vdev);
> if (!eth_dev) {
> PMD_INIT_LOG(ERR, "virtio_user fails to alloc device");
> goto end;
> }
>
> - hw = eth_dev->data->dev_private;
> - if (virtio_user_dev_init(hw->virtio_user_dev, path, queues, cq,
> + dev = eth_dev->data->dev_private;
> + hw = &dev->hw;
> + if (virtio_user_dev_init(dev, path, queues, cq,
> queue_size, mac_addr, &ifname, server_mode,
> mrg_rxbuf, in_order, packed_vq, backend_type) < 0) {
> PMD_INIT_LOG(ERR, "virtio_user_dev_init fails");
> @@ -912,7 +904,6 @@ static int virtio_user_pmd_dma_map(struct rte_vdev_device
> *vdev, void *addr,
> const char *name;
> struct rte_eth_dev *eth_dev;
> struct virtio_user_dev *dev;
> - struct virtio_hw *hw;
>
> if (!vdev)
> return -EINVAL;
> @@ -923,8 +914,7 @@ static int virtio_user_pmd_dma_map(struct rte_vdev_device
> *vdev, void *addr,
> if (!eth_dev)
> return 0;
>
> - hw = (struct virtio_hw *)eth_dev->data->dev_private;
> - dev = hw->virtio_user_dev;
> + dev = eth_dev->data->dev_private;
>
> if (dev->ops->dma_map)
> return dev->ops->dma_map(dev, addr, iova, len);
> @@ -938,7 +928,6 @@ static int virtio_user_pmd_dma_unmap(struct
> rte_vdev_device *vdev, void *addr,
> const char *name;
> struct rte_eth_dev *eth_dev;
> struct virtio_user_dev *dev;
> - struct virtio_hw *hw;
>
> if (!vdev)
> return -EINVAL;
> @@ -949,8 +938,7 @@ static int virtio_user_pmd_dma_unmap(struct
> rte_vdev_device *vdev, void *addr,
> if (!eth_dev)
> return 0;
>
> - hw = (struct virtio_hw *)eth_dev->data->dev_private;
> - dev = hw->virtio_user_dev;
> + dev = eth_dev->data->dev_private;
>
> if (dev->ops->dma_unmap)
> return dev->ops->dma_unmap(dev, addr, iova, len);
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device
2020-12-20 21:13 ` [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
@ 2021-01-05 21:16 ` David Marchand
2021-01-14 9:26 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:16 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves the virtio_hw structure into the virtio_user_dev
> structure, with the goal of making virtio_hw bus-agnostic.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Just one comment, the rest lgtm.
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> index 516d0ee577..1f1f63a1a5 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -26,13 +26,13 @@
> #include "virtio_user/virtio_user_dev.h"
> #include "virtio_user/vhost.h"
>
> -#define virtio_user_get_dev(hw) \
> - ((struct virtio_user_dev *)(hw)->virtio_user_dev)
> +#define virtio_user_get_dev(hw) container_of(hw, struct virtio_user_dev, hw)
Since the hw parameter is expanded as both the object and the field
name too, this macro prevents us from calling anything but
virtio_user_get_dev(hw).
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device
2021-01-05 21:16 ` David Marchand
@ 2021-01-14 9:26 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 9:26 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/5/21 10:16 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch moves the virtio_hw structure into the virtio_user_dev
>> structure, with the goal of making virtio_hw bus-agnostic.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
> Just one comment, the rest lgtm.
>
>
>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
>> index 516d0ee577..1f1f63a1a5 100644
>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>> @@ -26,13 +26,13 @@
>> #include "virtio_user/virtio_user_dev.h"
>> #include "virtio_user/vhost.h"
>>
>> -#define virtio_user_get_dev(hw) \
>> - ((struct virtio_user_dev *)(hw)->virtio_user_dev)
>> +#define virtio_user_get_dev(hw) container_of(hw, struct virtio_user_dev, hw)
>
> Since the hw parameter is expanded as both the object and the field
> name too, this macro prevents us from calling anything but
> virtio_user_get_dev(hw).
Indeed!
I propose to change the parameter to hwp (hw pointer).
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (2 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 03/40] net/virtio: refactor virtio-user device Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:16 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file Maxime Coquelin
` (36 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch initiate refactoring of Virtio PCI, by introducing
a new device structure for PCI-specific metadata.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 2 +-
drivers/net/virtio/virtio_pci.h | 5 +++++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 3ace25ac80..99a5a1bb88 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2151,7 +2151,7 @@ static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
if (vdpa == 1)
return 1;
- return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_hw),
+ return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
eth_virtio_dev_init);
}
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index b35a596169..8d3dc0e22e 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -289,6 +289,11 @@ struct virtio_hw {
struct virtqueue **vqs;
};
+struct virtio_pci_dev {
+ struct virtio_hw hw;
+};
+
+#define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
/*
* While virtio_hw is stored in shared memory, this structure stores
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2020-12-20 21:13 ` [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata Maxime Coquelin
@ 2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:16 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:13 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 04/40] net/virtio: introduce PCI device metadata
>
> This patch initiate refactoring of Virtio PCI, by introducing
> a new device structure for PCI-specific metadata.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 2 +-
> drivers/net/virtio/virtio_pci.h | 5 +++++
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 3ace25ac80..99a5a1bb88 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -2151,7 +2151,7 @@ static int eth_virtio_pci_probe(struct rte_pci_driver
> *pci_drv __rte_unused,
> if (vdpa == 1)
> return 1;
>
> - return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_hw),
> + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct
> virtio_pci_dev),
> eth_virtio_dev_init);
> }
>
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index b35a596169..8d3dc0e22e 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -289,6 +289,11 @@ struct virtio_hw {
> struct virtqueue **vqs;
> };
>
> +struct virtio_pci_dev {
> + struct virtio_hw hw;
> +};
> +
> +#define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
>
> /*
> * While virtio_hw is stored in shared memory, this structure stores
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2020-12-20 21:13 ` [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
@ 2021-01-05 21:16 ` David Marchand
2021-01-14 11:05 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:16 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch initiate refactoring of Virtio PCI, by introducing
> a new device structure for PCI-specific metadata.
This works, but this patch seems artificial.
The eth_virtio_dev_init expects dev->data->dev_private to be a virtio_hw object.
You can introduce this later in the series when really needed.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2021-01-05 21:16 ` David Marchand
@ 2021-01-14 11:05 ` Maxime Coquelin
2021-01-14 14:40 ` David Marchand
0 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 11:05 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/5/21 10:16 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch initiate refactoring of Virtio PCI, by introducing
>> a new device structure for PCI-specific metadata.
>
> This works, but this patch seems artificial.
>
> The eth_virtio_dev_init expects dev->data->dev_private to be a virtio_hw object.
> You can introduce this later in the series when really needed.
>
>
I propose to squash it into patch 5, which moves PCI specific init to a
dedicated file.
Is that OK for you?
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2021-01-14 11:05 ` Maxime Coquelin
@ 2021-01-14 14:40 ` David Marchand
2021-01-14 14:44 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-14 14:40 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Thu, Jan 14, 2021 at 12:05 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> On 1/5/21 10:16 PM, David Marchand wrote:
> > On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> > <maxime.coquelin@redhat.com> wrote:
> >>
> >> This patch initiate refactoring of Virtio PCI, by introducing
> >> a new device structure for PCI-specific metadata.
> >
> > This works, but this patch seems artificial.
> >
> > The eth_virtio_dev_init expects dev->data->dev_private to be a virtio_hw object.
> > You can introduce this later in the series when really needed.
> >
> >
> I propose to squash it into patch 5, which moves PCI specific init to a
> dedicated file.
>
> Is that OK for you?
Or leave this commit like this but add a check on the virtio_hw field
being the first field of the bus specific objects.
Something like:
RTE_BUILD_BUG_ON(offsetof(struct virtio_pci_dev, hw) != 0);
+ the same with virtio_user in the relevant patch.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata
2021-01-14 14:40 ` David Marchand
@ 2021-01-14 14:44 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 14:44 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/14/21 3:40 PM, David Marchand wrote:
> On Thu, Jan 14, 2021 at 12:05 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> On 1/5/21 10:16 PM, David Marchand wrote:
>>> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
>>> <maxime.coquelin@redhat.com> wrote:
>>>>
>>>> This patch initiate refactoring of Virtio PCI, by introducing
>>>> a new device structure for PCI-specific metadata.
>>>
>>> This works, but this patch seems artificial.
>>>
>>> The eth_virtio_dev_init expects dev->data->dev_private to be a virtio_hw object.
>>> You can introduce this later in the series when really needed.
>>>
>>>
>> I propose to squash it into patch 5, which moves PCI specific init to a
>> dedicated file.
>>
>> Is that OK for you?
>
> Or leave this commit like this but add a check on the virtio_hw field
> being the first field of the bus specific objects.
> Something like:
> RTE_BUILD_BUG_ON(offsetof(struct virtio_pci_dev, hw) != 0);
>
> + the same with virtio_user in the relevant patch.
>
I like the idea, will be done in v2.
Thanks for the suggestion,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (3 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 04/40] net/virtio: introduce PCI device metadata Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:19 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init Maxime Coquelin
` (35 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Thi patch moves the PCI Ethernet device registration bits
in a dedicated patch. In following patches, more code will
be moved there, with the goal of making virtio_ethdev.c
truely bus-agnostic.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/meson.build | 1 +
drivers/net/virtio/virtio_ethdev.c | 114 +------------------
drivers/net/virtio/virtio_ethdev.h | 2 +
drivers/net/virtio/virtio_pci_ethdev.c | 148 +++++++++++++++++++++++++
4 files changed, 156 insertions(+), 109 deletions(-)
create mode 100644 drivers/net/virtio/virtio_pci_ethdev.c
diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
index eaed46373d..8e0f1a9951 100644
--- a/drivers/net/virtio/meson.build
+++ b/drivers/net/virtio/meson.build
@@ -2,6 +2,7 @@
# Copyright(c) 2018 Intel Corporation
sources += files('virtio_ethdev.c',
+ 'virtio_pci_ethdev.c',
'virtio_pci.c',
'virtio_rxtx.c',
'virtio_rxtx_simple.c',
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 99a5a1bb88..ca21a019e5 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -38,17 +38,14 @@
#include "virtio_rxtx.h"
#include "virtio_user/virtio_user_dev.h"
-static int eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev);
static int virtio_dev_configure(struct rte_eth_dev *dev);
static int virtio_dev_start(struct rte_eth_dev *dev);
-static int virtio_dev_stop(struct rte_eth_dev *dev);
static int virtio_dev_promiscuous_enable(struct rte_eth_dev *dev);
static int virtio_dev_promiscuous_disable(struct rte_eth_dev *dev);
static int virtio_dev_allmulticast_enable(struct rte_eth_dev *dev);
static int virtio_dev_allmulticast_disable(struct rte_eth_dev *dev);
static uint32_t virtio_dev_speed_capa_get(uint32_t speed);
static int virtio_dev_devargs_parse(struct rte_devargs *devargs,
- int *vdpa,
uint32_t *speed,
int *vectorized);
static int virtio_dev_info_get(struct rte_eth_dev *dev,
@@ -89,15 +86,6 @@ static int virtio_dev_queue_stats_mapping_set(
static void virtio_notify_peers(struct rte_eth_dev *dev);
static void virtio_ack_link_announce(struct rte_eth_dev *dev);
-/*
- * The set of PCI devices this driver supports
- */
-static const struct rte_pci_id pci_id_virtio_map[] = {
- { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
- { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
- { .vendor_id = 0, /* sentinel */ },
-};
-
struct rte_virtio_xstats_name_off {
char name[RTE_ETH_XSTATS_NAME_SIZE];
unsigned offset;
@@ -714,7 +702,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
-static int
+int
virtio_dev_close(struct rte_eth_dev *dev)
{
struct virtio_hw *hw = dev->data->dev_private;
@@ -1929,8 +1917,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
return 0;
}
- ret = virtio_dev_devargs_parse(eth_dev->device->devargs,
- NULL, &speed, &vectorized);
+ ret = virtio_dev_devargs_parse(eth_dev->device->devargs, &speed, &vectorized);
if (ret < 0)
return ret;
hw->speed = speed;
@@ -1992,36 +1979,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
return ret;
}
-static int
-eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
-{
- int ret;
- PMD_INIT_FUNC_TRACE();
-
- if (rte_eal_process_type() == RTE_PROC_SECONDARY)
- return 0;
-
- ret = virtio_dev_stop(eth_dev);
- virtio_dev_close(eth_dev);
-
- PMD_INIT_LOG(DEBUG, "dev_uninit completed");
-
- return ret;
-}
-
-
-static int vdpa_check_handler(__rte_unused const char *key,
- const char *value, void *ret_val)
-{
- if (strcmp(value, "1") == 0)
- *(int *)ret_val = 1;
- else
- *(int *)ret_val = 0;
-
- return 0;
-}
-
-
static uint32_t
virtio_dev_speed_capa_get(uint32_t speed)
{
@@ -2059,10 +2016,8 @@ static int vectorized_check_handler(__rte_unused const char *key,
}
#define VIRTIO_ARG_SPEED "speed"
-#define VIRTIO_ARG_VDPA "vdpa"
#define VIRTIO_ARG_VECTORIZED "vectorized"
-
static int
link_speed_handler(const char *key __rte_unused,
const char *value, void *ret_val)
@@ -2081,8 +2036,7 @@ link_speed_handler(const char *key __rte_unused,
static int
-virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
- uint32_t *speed, int *vectorized)
+virtio_dev_devargs_parse(struct rte_devargs *devargs, uint32_t *speed, int *vectorized)
{
struct rte_kvargs *kvlist;
int ret = 0;
@@ -2095,18 +2049,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
PMD_INIT_LOG(ERR, "error when parsing param");
return 0;
}
- if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
- /* vdpa mode selected when there's a key-value pair:
- * vdpa=1
- */
- ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
- vdpa_check_handler, vdpa);
- if (ret < 0) {
- PMD_INIT_LOG(ERR, "Failed to parse %s",
- VIRTIO_ARG_VDPA);
- goto exit;
- }
- }
+
if (speed && rte_kvargs_count(kvlist, VIRTIO_ARG_SPEED) == 1) {
ret = rte_kvargs_process(kvlist,
VIRTIO_ARG_SPEED,
@@ -2135,52 +2078,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
return ret;
}
-static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
- struct rte_pci_device *pci_dev)
-{
- int vdpa = 0;
- int ret = 0;
-
- ret = virtio_dev_devargs_parse(pci_dev->device.devargs, &vdpa, NULL,
- NULL);
- if (ret < 0) {
- PMD_INIT_LOG(ERR, "devargs parsing is failed");
- return ret;
- }
- /* virtio pmd skips probe if device needs to work in vdpa mode */
- if (vdpa == 1)
- return 1;
-
- return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
- eth_virtio_dev_init);
-}
-
-static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
-{
- int ret;
-
- ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_dev_uninit);
- /* Port has already been released by close. */
- if (ret == -ENODEV)
- ret = 0;
- return ret;
-}
-
-static struct rte_pci_driver rte_virtio_pmd = {
- .driver = {
- .name = "net_virtio",
- },
- .id_table = pci_id_virtio_map,
- .drv_flags = 0,
- .probe = eth_virtio_pci_probe,
- .remove = eth_virtio_pci_remove,
-};
-RTE_INIT(rte_virtio_pmd_init)
-{
- rte_eal_iopl_init();
- rte_pci_register(&rte_virtio_pmd);
-}
static bool
rx_offload_enabled(struct virtio_hw *hw)
@@ -2521,7 +2419,7 @@ static void virtio_dev_free_mbufs(struct rte_eth_dev *dev)
/*
* Stop device: disable interrupt and mark link down
*/
-static int
+int
virtio_dev_stop(struct rte_eth_dev *dev)
{
struct virtio_hw *hw = dev->data->dev_private;
@@ -2673,7 +2571,5 @@ __rte_unused uint8_t is_rx)
}
RTE_PMD_EXPORT_NAME(net_virtio, __COUNTER__);
-RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
-RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_LOG_REGISTER(virtio_logtype_init, pmd.net.virtio.init, NOTICE);
RTE_LOG_REGISTER(virtio_logtype_driver, pmd.net.virtio.driver, NOTICE);
diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
index b7d52d497f..13395937c8 100644
--- a/drivers/net/virtio/virtio_ethdev.h
+++ b/drivers/net/virtio/virtio_ethdev.h
@@ -117,6 +117,8 @@ void virtio_interrupt_handler(void *param);
int virtio_dev_pause(struct rte_eth_dev *dev);
void virtio_dev_resume(struct rte_eth_dev *dev);
+int virtio_dev_stop(struct rte_eth_dev *dev);
+int virtio_dev_close(struct rte_eth_dev *dev);
int virtio_inject_pkts(struct rte_eth_dev *dev, struct rte_mbuf **tx_pkts,
int nb_pkts);
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
new file mode 100644
index 0000000000..9c0935068e
--- /dev/null
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -0,0 +1,148 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2016 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+#include <errno.h>
+#include <unistd.h>
+
+#include <rte_ethdev_driver.h>
+#include <rte_ethdev_pci.h>
+#include <rte_pci.h>
+#include <rte_bus_pci.h>
+#include <rte_errno.h>
+
+#include <rte_memory.h>
+#include <rte_eal.h>
+#include <rte_dev.h>
+#include <rte_kvargs.h>
+
+#include "virtio_ethdev.h"
+#include "virtio_pci.h"
+#include "virtio_logs.h"
+
+/*
+ * The set of PCI devices this driver supports
+ */
+static const struct rte_pci_id pci_id_virtio_map[] = {
+ { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
+ { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+static int
+eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
+{
+ return eth_virtio_dev_init(eth_dev);
+}
+
+static int
+eth_virtio_pci_uninit(struct rte_eth_dev *eth_dev)
+{
+ int ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (rte_eal_process_type() == RTE_PROC_SECONDARY)
+ return 0;
+
+ ret = virtio_dev_stop(eth_dev);
+ virtio_dev_close(eth_dev);
+
+ PMD_INIT_LOG(DEBUG, "dev_uninit completed");
+
+ return ret;
+}
+
+static int vdpa_check_handler(__rte_unused const char *key,
+ const char *value, void *ret_val)
+{
+ if (strcmp(value, "1") == 0)
+ *(int *)ret_val = 1;
+ else
+ *(int *)ret_val = 0;
+
+ return 0;
+}
+
+#define VIRTIO_ARG_VDPA "vdpa"
+
+static int
+virtio_pci_devargs_parse(struct rte_devargs *devargs, int *vdpa)
+{
+ struct rte_kvargs *kvlist;
+ int ret = 0;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "error when parsing param");
+ return 0;
+ }
+
+ if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
+ /* vdpa mode selected when there's a key-value pair:
+ * vdpa=1
+ */
+ ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
+ vdpa_check_handler, vdpa);
+ if (ret < 0)
+ PMD_INIT_LOG(ERR, "Failed to parse %s", VIRTIO_ARG_VDPA);
+ }
+
+ rte_kvargs_free(kvlist);
+
+ return ret;
+}
+
+static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int vdpa = 0;
+ int ret = 0;
+
+ ret = virtio_pci_devargs_parse(pci_dev->device.devargs, &vdpa);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "devargs parsing is failed");
+ return ret;
+ }
+ /* virtio pmd skips probe if device needs to work in vdpa mode */
+ if (vdpa == 1)
+ return 1;
+
+ return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
+ eth_virtio_pci_init);
+}
+
+static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
+{
+ int ret;
+
+ ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_pci_uninit);
+ /* Port has already been released by close. */
+ if (ret == -ENODEV)
+ ret = 0;
+ return ret;
+}
+
+static struct rte_pci_driver rte_virtio_pmd = {
+ .driver = {
+ .name = "net_virtio",
+ },
+ .id_table = pci_id_virtio_map,
+ .drv_flags = 0,
+ .probe = eth_virtio_pci_probe,
+ .remove = eth_virtio_pci_remove,
+};
+
+RTE_INIT(rte_virtio_pmd_init)
+{
+ rte_eal_iopl_init();
+ rte_pci_register(&rte_virtio_pmd);
+}
+
+RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file
2020-12-20 21:13 ` [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file Maxime Coquelin
@ 2020-12-30 3:02 ` Xia, Chenbo
2021-01-05 21:19 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 05/40] net/virtio: move PCI device init in dedicated file
>
> Thi patch moves the PCI Ethernet device registration bits
s/Thi/This
> in a dedicated patch. In following patches, more code will
> be moved there, with the goal of making virtio_ethdev.c
> truely bus-agnostic.
s/truely/truly
With above fixes:
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/meson.build | 1 +
> drivers/net/virtio/virtio_ethdev.c | 114 +------------------
> drivers/net/virtio/virtio_ethdev.h | 2 +
> drivers/net/virtio/virtio_pci_ethdev.c | 148 +++++++++++++++++++++++++
> 4 files changed, 156 insertions(+), 109 deletions(-)
> create mode 100644 drivers/net/virtio/virtio_pci_ethdev.c
>
> diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
> index eaed46373d..8e0f1a9951 100644
> --- a/drivers/net/virtio/meson.build
> +++ b/drivers/net/virtio/meson.build
> @@ -2,6 +2,7 @@
> # Copyright(c) 2018 Intel Corporation
>
> sources += files('virtio_ethdev.c',
> + 'virtio_pci_ethdev.c',
> 'virtio_pci.c',
> 'virtio_rxtx.c',
> 'virtio_rxtx_simple.c',
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 99a5a1bb88..ca21a019e5 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -38,17 +38,14 @@
> #include "virtio_rxtx.h"
> #include "virtio_user/virtio_user_dev.h"
>
> -static int eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev);
> static int virtio_dev_configure(struct rte_eth_dev *dev);
> static int virtio_dev_start(struct rte_eth_dev *dev);
> -static int virtio_dev_stop(struct rte_eth_dev *dev);
> static int virtio_dev_promiscuous_enable(struct rte_eth_dev *dev);
> static int virtio_dev_promiscuous_disable(struct rte_eth_dev *dev);
> static int virtio_dev_allmulticast_enable(struct rte_eth_dev *dev);
> static int virtio_dev_allmulticast_disable(struct rte_eth_dev *dev);
> static uint32_t virtio_dev_speed_capa_get(uint32_t speed);
> static int virtio_dev_devargs_parse(struct rte_devargs *devargs,
> - int *vdpa,
> uint32_t *speed,
> int *vectorized);
> static int virtio_dev_info_get(struct rte_eth_dev *dev,
> @@ -89,15 +86,6 @@ static int virtio_dev_queue_stats_mapping_set(
> static void virtio_notify_peers(struct rte_eth_dev *dev);
> static void virtio_ack_link_announce(struct rte_eth_dev *dev);
>
> -/*
> - * The set of PCI devices this driver supports
> - */
> -static const struct rte_pci_id pci_id_virtio_map[] = {
> - { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
> - { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
> - { .vendor_id = 0, /* sentinel */ },
> -};
> -
> struct rte_virtio_xstats_name_off {
> char name[RTE_ETH_XSTATS_NAME_SIZE];
> unsigned offset;
> @@ -714,7 +702,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
>
> static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
>
> -static int
> +int
> virtio_dev_close(struct rte_eth_dev *dev)
> {
> struct virtio_hw *hw = dev->data->dev_private;
> @@ -1929,8 +1917,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>
> return 0;
> }
> - ret = virtio_dev_devargs_parse(eth_dev->device->devargs,
> - NULL, &speed, &vectorized);
> + ret = virtio_dev_devargs_parse(eth_dev->device->devargs, &speed,
> &vectorized);
> if (ret < 0)
> return ret;
> hw->speed = speed;
> @@ -1992,36 +1979,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> return ret;
> }
>
> -static int
> -eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
> -{
> - int ret;
> - PMD_INIT_FUNC_TRACE();
> -
> - if (rte_eal_process_type() == RTE_PROC_SECONDARY)
> - return 0;
> -
> - ret = virtio_dev_stop(eth_dev);
> - virtio_dev_close(eth_dev);
> -
> - PMD_INIT_LOG(DEBUG, "dev_uninit completed");
> -
> - return ret;
> -}
> -
> -
> -static int vdpa_check_handler(__rte_unused const char *key,
> - const char *value, void *ret_val)
> -{
> - if (strcmp(value, "1") == 0)
> - *(int *)ret_val = 1;
> - else
> - *(int *)ret_val = 0;
> -
> - return 0;
> -}
> -
> -
> static uint32_t
> virtio_dev_speed_capa_get(uint32_t speed)
> {
> @@ -2059,10 +2016,8 @@ static int vectorized_check_handler(__rte_unused const
> char *key,
> }
>
> #define VIRTIO_ARG_SPEED "speed"
> -#define VIRTIO_ARG_VDPA "vdpa"
> #define VIRTIO_ARG_VECTORIZED "vectorized"
>
> -
> static int
> link_speed_handler(const char *key __rte_unused,
> const char *value, void *ret_val)
> @@ -2081,8 +2036,7 @@ link_speed_handler(const char *key __rte_unused,
>
>
> static int
> -virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
> - uint32_t *speed, int *vectorized)
> +virtio_dev_devargs_parse(struct rte_devargs *devargs, uint32_t *speed, int
> *vectorized)
> {
> struct rte_kvargs *kvlist;
> int ret = 0;
> @@ -2095,18 +2049,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs,
> int *vdpa,
> PMD_INIT_LOG(ERR, "error when parsing param");
> return 0;
> }
> - if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
> - /* vdpa mode selected when there's a key-value pair:
> - * vdpa=1
> - */
> - ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
> - vdpa_check_handler, vdpa);
> - if (ret < 0) {
> - PMD_INIT_LOG(ERR, "Failed to parse %s",
> - VIRTIO_ARG_VDPA);
> - goto exit;
> - }
> - }
> +
> if (speed && rte_kvargs_count(kvlist, VIRTIO_ARG_SPEED) == 1) {
> ret = rte_kvargs_process(kvlist,
> VIRTIO_ARG_SPEED,
> @@ -2135,52 +2078,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs,
> int *vdpa,
> return ret;
> }
>
> -static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> - struct rte_pci_device *pci_dev)
> -{
> - int vdpa = 0;
> - int ret = 0;
> -
> - ret = virtio_dev_devargs_parse(pci_dev->device.devargs, &vdpa, NULL,
> - NULL);
> - if (ret < 0) {
> - PMD_INIT_LOG(ERR, "devargs parsing is failed");
> - return ret;
> - }
> - /* virtio pmd skips probe if device needs to work in vdpa mode */
> - if (vdpa == 1)
> - return 1;
> -
> - return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct
> virtio_pci_dev),
> - eth_virtio_dev_init);
> -}
> -
> -static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
> -{
> - int ret;
> -
> - ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_dev_uninit);
> - /* Port has already been released by close. */
> - if (ret == -ENODEV)
> - ret = 0;
> - return ret;
> -}
> -
> -static struct rte_pci_driver rte_virtio_pmd = {
> - .driver = {
> - .name = "net_virtio",
> - },
> - .id_table = pci_id_virtio_map,
> - .drv_flags = 0,
> - .probe = eth_virtio_pci_probe,
> - .remove = eth_virtio_pci_remove,
> -};
>
> -RTE_INIT(rte_virtio_pmd_init)
> -{
> - rte_eal_iopl_init();
> - rte_pci_register(&rte_virtio_pmd);
> -}
>
> static bool
> rx_offload_enabled(struct virtio_hw *hw)
> @@ -2521,7 +2419,7 @@ static void virtio_dev_free_mbufs(struct rte_eth_dev
> *dev)
> /*
> * Stop device: disable interrupt and mark link down
> */
> -static int
> +int
> virtio_dev_stop(struct rte_eth_dev *dev)
> {
> struct virtio_hw *hw = dev->data->dev_private;
> @@ -2673,7 +2571,5 @@ __rte_unused uint8_t is_rx)
> }
>
> RTE_PMD_EXPORT_NAME(net_virtio, __COUNTER__);
> -RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
> -RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-
> pci");
> RTE_LOG_REGISTER(virtio_logtype_init, pmd.net.virtio.init, NOTICE);
> RTE_LOG_REGISTER(virtio_logtype_driver, pmd.net.virtio.driver, NOTICE);
> diff --git a/drivers/net/virtio/virtio_ethdev.h
> b/drivers/net/virtio/virtio_ethdev.h
> index b7d52d497f..13395937c8 100644
> --- a/drivers/net/virtio/virtio_ethdev.h
> +++ b/drivers/net/virtio/virtio_ethdev.h
> @@ -117,6 +117,8 @@ void virtio_interrupt_handler(void *param);
>
> int virtio_dev_pause(struct rte_eth_dev *dev);
> void virtio_dev_resume(struct rte_eth_dev *dev);
> +int virtio_dev_stop(struct rte_eth_dev *dev);
> +int virtio_dev_close(struct rte_eth_dev *dev);
> int virtio_inject_pkts(struct rte_eth_dev *dev, struct rte_mbuf **tx_pkts,
> int nb_pkts);
>
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> new file mode 100644
> index 0000000000..9c0935068e
> --- /dev/null
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -0,0 +1,148 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2016 Intel Corporation
> + */
> +
> +#include <stdint.h>
> +#include <string.h>
> +#include <stdio.h>
> +#include <errno.h>
> +#include <unistd.h>
> +
> +#include <rte_ethdev_driver.h>
> +#include <rte_ethdev_pci.h>
> +#include <rte_pci.h>
> +#include <rte_bus_pci.h>
> +#include <rte_errno.h>
> +
> +#include <rte_memory.h>
> +#include <rte_eal.h>
> +#include <rte_dev.h>
> +#include <rte_kvargs.h>
> +
> +#include "virtio_ethdev.h"
> +#include "virtio_pci.h"
> +#include "virtio_logs.h"
> +
> +/*
> + * The set of PCI devices this driver supports
> + */
> +static const struct rte_pci_id pci_id_virtio_map[] = {
> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +static int
> +eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> +{
> + return eth_virtio_dev_init(eth_dev);
> +}
> +
> +static int
> +eth_virtio_pci_uninit(struct rte_eth_dev *eth_dev)
> +{
> + int ret;
> + PMD_INIT_FUNC_TRACE();
> +
> + if (rte_eal_process_type() == RTE_PROC_SECONDARY)
> + return 0;
> +
> + ret = virtio_dev_stop(eth_dev);
> + virtio_dev_close(eth_dev);
> +
> + PMD_INIT_LOG(DEBUG, "dev_uninit completed");
> +
> + return ret;
> +}
> +
> +static int vdpa_check_handler(__rte_unused const char *key,
> + const char *value, void *ret_val)
> +{
> + if (strcmp(value, "1") == 0)
> + *(int *)ret_val = 1;
> + else
> + *(int *)ret_val = 0;
> +
> + return 0;
> +}
> +
> +#define VIRTIO_ARG_VDPA "vdpa"
> +
> +static int
> +virtio_pci_devargs_parse(struct rte_devargs *devargs, int *vdpa)
> +{
> + struct rte_kvargs *kvlist;
> + int ret = 0;
> +
> + if (devargs == NULL)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, NULL);
> + if (kvlist == NULL) {
> + PMD_INIT_LOG(ERR, "error when parsing param");
> + return 0;
> + }
> +
> + if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
> + /* vdpa mode selected when there's a key-value pair:
> + * vdpa=1
> + */
> + ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
> + vdpa_check_handler, vdpa);
> + if (ret < 0)
> + PMD_INIT_LOG(ERR, "Failed to parse %s", VIRTIO_ARG_VDPA);
> + }
> +
> + rte_kvargs_free(kvlist);
> +
> + return ret;
> +}
> +
> +static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev)
> +{
> + int vdpa = 0;
> + int ret = 0;
> +
> + ret = virtio_pci_devargs_parse(pci_dev->device.devargs, &vdpa);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "devargs parsing is failed");
> + return ret;
> + }
> + /* virtio pmd skips probe if device needs to work in vdpa mode */
> + if (vdpa == 1)
> + return 1;
> +
> + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct
> virtio_pci_dev),
> + eth_virtio_pci_init);
> +}
> +
> +static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
> +{
> + int ret;
> +
> + ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_pci_uninit);
> + /* Port has already been released by close. */
> + if (ret == -ENODEV)
> + ret = 0;
> + return ret;
> +}
> +
> +static struct rte_pci_driver rte_virtio_pmd = {
> + .driver = {
> + .name = "net_virtio",
> + },
> + .id_table = pci_id_virtio_map,
> + .drv_flags = 0,
> + .probe = eth_virtio_pci_probe,
> + .remove = eth_virtio_pci_remove,
> +};
> +
> +RTE_INIT(rte_virtio_pmd_init)
> +{
> + rte_eal_iopl_init();
> + rte_pci_register(&rte_virtio_pmd);
> +}
> +
> +RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-
> pci");
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file
2020-12-20 21:13 ` [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file Maxime Coquelin
2020-12-30 3:02 ` Xia, Chenbo
@ 2021-01-05 21:19 ` David Marchand
2021-01-14 16:04 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:19 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index 99a5a1bb88..ca21a019e5 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
[...]
> @@ -2135,52 +2078,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
> return ret;
> }
>
> -static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> - struct rte_pci_device *pci_dev)
> -{
> - int vdpa = 0;
> - int ret = 0;
> -
> - ret = virtio_dev_devargs_parse(pci_dev->device.devargs, &vdpa, NULL,
> - NULL);
> - if (ret < 0) {
> - PMD_INIT_LOG(ERR, "devargs parsing is failed");
> - return ret;
> - }
> - /* virtio pmd skips probe if device needs to work in vdpa mode */
> - if (vdpa == 1)
> - return 1;
> -
> - return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
> - eth_virtio_dev_init);
> -}
> -
> -static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
> -{
> - int ret;
> -
> - ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_dev_uninit);
> - /* Port has already been released by close. */
> - if (ret == -ENODEV)
> - ret = 0;
> - return ret;
> -}
> -
> -static struct rte_pci_driver rte_virtio_pmd = {
> - .driver = {
> - .name = "net_virtio",
> - },
> - .id_table = pci_id_virtio_map,
> - .drv_flags = 0,
> - .probe = eth_virtio_pci_probe,
> - .remove = eth_virtio_pci_remove,
> -};
>
> -RTE_INIT(rte_virtio_pmd_init)
> -{
> - rte_eal_iopl_init();
> - rte_pci_register(&rte_virtio_pmd);
> -}
>
Leaving three empty lines if I count correctly.
> static bool
> rx_offload_enabled(struct virtio_hw *hw)
> @@ -2521,7 +2419,7 @@ static void virtio_dev_free_mbufs(struct rte_eth_dev *dev)
> /*
> * Stop device: disable interrupt and mark link down
> */
> -static int
> +int
> virtio_dev_stop(struct rte_eth_dev *dev)
> {
> struct virtio_hw *hw = dev->data->dev_private;
> @@ -2673,7 +2571,5 @@ __rte_unused uint8_t is_rx)
> }
>
> RTE_PMD_EXPORT_NAME(net_virtio, __COUNTER__);
This belongs with the rest of the pci driver declarations.
$ ./usertools/dpdk-pmdinfo.py
$HOME/builds/build-gcc-shared/drivers/librte_net_virtio.so
PMD NAME: net_virtio_user
PMD PARAMETERS: path=<path> mac=<mac addr> cq=<int> queue_size=<int>
queues=<int> iface=<string> server=<0|1> mrg_rxbuf=<0|1>
in_order=<0|1> packed_vq=<0|1> speed=<int> vectorized=<0|1>
PMD NAME: net_virtio
^^^
Here, I would expect the pci table and the kmod dependencies.
This makes me realise that the net_virtio_user driver is not exported
(but it seems to work without it.. interesting).
> -RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
> -RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
> RTE_LOG_REGISTER(virtio_logtype_init, pmd.net.virtio.init, NOTICE);
> RTE_LOG_REGISTER(virtio_logtype_driver, pmd.net.virtio.driver, NOTICE);
> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
> index b7d52d497f..13395937c8 100644
> --- a/drivers/net/virtio/virtio_ethdev.h
> +++ b/drivers/net/virtio/virtio_ethdev.h
> @@ -117,6 +117,8 @@ void virtio_interrupt_handler(void *param);
>
> int virtio_dev_pause(struct rte_eth_dev *dev);
> void virtio_dev_resume(struct rte_eth_dev *dev);
> +int virtio_dev_stop(struct rte_eth_dev *dev);
> +int virtio_dev_close(struct rte_eth_dev *dev);
> int virtio_inject_pkts(struct rte_eth_dev *dev, struct rte_mbuf **tx_pkts,
> int nb_pkts);
>
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
> new file mode 100644
> index 0000000000..9c0935068e
> --- /dev/null
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -0,0 +1,148 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2016 Intel Corporation
> + */
> +
> +#include <stdint.h>
> +#include <string.h>
> +#include <stdio.h>
> +#include <errno.h>
> +#include <unistd.h>
> +
> +#include <rte_ethdev_driver.h>
> +#include <rte_ethdev_pci.h>
> +#include <rte_pci.h>
> +#include <rte_bus_pci.h>
> +#include <rte_errno.h>
> +
> +#include <rte_memory.h>
> +#include <rte_eal.h>
> +#include <rte_dev.h>
> +#include <rte_kvargs.h>
> +
> +#include "virtio_ethdev.h"
> +#include "virtio_pci.h"
> +#include "virtio_logs.h"
> +
> +/*
> + * The set of PCI devices this driver supports
> + */
> +static const struct rte_pci_id pci_id_virtio_map[] = {
> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +static int
> +eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> +{
> + return eth_virtio_dev_init(eth_dev);
> +}
> +
> +static int
> +eth_virtio_pci_uninit(struct rte_eth_dev *eth_dev)
> +{
> + int ret;
> + PMD_INIT_FUNC_TRACE();
> +
> + if (rte_eal_process_type() == RTE_PROC_SECONDARY)
> + return 0;
> +
> + ret = virtio_dev_stop(eth_dev);
> + virtio_dev_close(eth_dev);
> +
> + PMD_INIT_LOG(DEBUG, "dev_uninit completed");
> +
> + return ret;
> +}
> +
> +static int vdpa_check_handler(__rte_unused const char *key,
> + const char *value, void *ret_val)
> +{
> + if (strcmp(value, "1") == 0)
> + *(int *)ret_val = 1;
> + else
> + *(int *)ret_val = 0;
> +
> + return 0;
> +}
> +
> +#define VIRTIO_ARG_VDPA "vdpa"
> +
> +static int
> +virtio_pci_devargs_parse(struct rte_devargs *devargs, int *vdpa)
> +{
> + struct rte_kvargs *kvlist;
> + int ret = 0;
> +
> + if (devargs == NULL)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, NULL);
> + if (kvlist == NULL) {
> + PMD_INIT_LOG(ERR, "error when parsing param");
> + return 0;
> + }
> +
> + if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
I would remove this check on vdpa pointer being NULL, as it gets
passed on the stack by a single caller in this file.
> + /* vdpa mode selected when there's a key-value pair:
> + * vdpa=1
> + */
> + ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
> + vdpa_check_handler, vdpa);
> + if (ret < 0)
> + PMD_INIT_LOG(ERR, "Failed to parse %s", VIRTIO_ARG_VDPA);
> + }
> +
> + rte_kvargs_free(kvlist);
> +
> + return ret;
> +}
> +
> +static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev)
> +{
> + int vdpa = 0;
> + int ret = 0;
> +
> + ret = virtio_pci_devargs_parse(pci_dev->device.devargs, &vdpa);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "devargs parsing is failed");
> + return ret;
> + }
> + /* virtio pmd skips probe if device needs to work in vdpa mode */
> + if (vdpa == 1)
> + return 1;
> +
> + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
> + eth_virtio_pci_init);
> +}
> +
> +static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
> +{
> + int ret;
> +
> + ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_pci_uninit);
> + /* Port has already been released by close. */
> + if (ret == -ENODEV)
> + ret = 0;
> + return ret;
> +}
> +
> +static struct rte_pci_driver rte_virtio_pmd = {
This seems too generic.
virtio_pci_pmd ?
> + .driver = {
> + .name = "net_virtio",
> + },
> + .id_table = pci_id_virtio_map,
> + .drv_flags = 0,
> + .probe = eth_virtio_pci_probe,
> + .remove = eth_virtio_pci_remove,
> +};
> +
> +RTE_INIT(rte_virtio_pmd_init)
virtio_pci_pmd_init ?
> +{
> + rte_eal_iopl_init();
> + rte_pci_register(&rte_virtio_pmd);
> +}
> +
> +RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file
2021-01-05 21:19 ` David Marchand
@ 2021-01-14 16:04 ` Maxime Coquelin
2021-01-14 16:14 ` David Marchand
0 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 16:04 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/5/21 10:19 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
>> index 99a5a1bb88..ca21a019e5 100644
>> --- a/drivers/net/virtio/virtio_ethdev.c
>> +++ b/drivers/net/virtio/virtio_ethdev.c
>
> [...]
>
>> @@ -2135,52 +2078,7 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs, int *vdpa,
>> return ret;
>> }
>>
>> -static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>> - struct rte_pci_device *pci_dev)
>> -{
>> - int vdpa = 0;
>> - int ret = 0;
>> -
>> - ret = virtio_dev_devargs_parse(pci_dev->device.devargs, &vdpa, NULL,
>> - NULL);
>> - if (ret < 0) {
>> - PMD_INIT_LOG(ERR, "devargs parsing is failed");
>> - return ret;
>> - }
>> - /* virtio pmd skips probe if device needs to work in vdpa mode */
>> - if (vdpa == 1)
>> - return 1;
>> -
>> - return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
>> - eth_virtio_dev_init);
>> -}
>> -
>> -static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
>> -{
>> - int ret;
>> -
>> - ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_dev_uninit);
>> - /* Port has already been released by close. */
>> - if (ret == -ENODEV)
>> - ret = 0;
>> - return ret;
>> -}
>> -
>> -static struct rte_pci_driver rte_virtio_pmd = {
>> - .driver = {
>> - .name = "net_virtio",
>> - },
>> - .id_table = pci_id_virtio_map,
>> - .drv_flags = 0,
>> - .probe = eth_virtio_pci_probe,
>> - .remove = eth_virtio_pci_remove,
>> -};
>>
>> -RTE_INIT(rte_virtio_pmd_init)
>> -{
>> - rte_eal_iopl_init();
>> - rte_pci_register(&rte_virtio_pmd);
>> -}
>>
>
> Leaving three empty lines if I count correctly.
>
>
>> static bool
>> rx_offload_enabled(struct virtio_hw *hw)
>> @@ -2521,7 +2419,7 @@ static void virtio_dev_free_mbufs(struct rte_eth_dev *dev)
>> /*
>> * Stop device: disable interrupt and mark link down
>> */
>> -static int
>> +int
>> virtio_dev_stop(struct rte_eth_dev *dev)
>> {
>> struct virtio_hw *hw = dev->data->dev_private;
>> @@ -2673,7 +2571,5 @@ __rte_unused uint8_t is_rx)
>> }
>>
>> RTE_PMD_EXPORT_NAME(net_virtio, __COUNTER__);
>
> This belongs with the rest of the pci driver declarations.
>
> $ ./usertools/dpdk-pmdinfo.py
> $HOME/builds/build-gcc-shared/drivers/librte_net_virtio.so
> PMD NAME: net_virtio_user
> PMD PARAMETERS: path=<path> mac=<mac addr> cq=<int> queue_size=<int>
> queues=<int> iface=<string> server=<0|1> mrg_rxbuf=<0|1>
> in_order=<0|1> packed_vq=<0|1> speed=<int> vectorized=<0|1>
>
> PMD NAME: net_virtio
>
> ^^^
> Here, I would expect the pci table and the kmod dependencies.
>
>
> This makes me realise that the net_virtio_user driver is not exported
> (but it seems to work without it.. interesting).
As discussed offlist, I have moved RTE_PMD_EXPORT_NAME to
virtio_pci_ethdev.c, and I now get expected output:
$ ./usertools/dpdk-pmdinfo.py build/drivers/librte_net_virtio.so
PMD NAME: net_virtio_user
PMD PARAMETERS: path=<path> mac=<mac addr> cq=<int> queue_size=<int>
queues=<int> iface=<string> server=<0|1> mrg_rxbuf=<0|1> in_order=<0|1>
packed_vq=<0|1> speed=<int> vectorized=<0|1>
PMD NAME: net_virtio
PMD KMOD DEPENDENCIES: * igb_uio | uio_pci_generic | vfio-pci
PMD HW SUPPORT:
Red Hat, Inc. (1af4) : Virtio network device (1000) (All Subdevices)
Red Hat, Inc. (1af4) : Virtio network device (1041) (All Subdevices)
>
>> -RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
>> -RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
>> RTE_LOG_REGISTER(virtio_logtype_init, pmd.net.virtio.init, NOTICE);
>> RTE_LOG_REGISTER(virtio_logtype_driver, pmd.net.virtio.driver, NOTICE);
>> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
>> index b7d52d497f..13395937c8 100644
>> --- a/drivers/net/virtio/virtio_ethdev.h
>> +++ b/drivers/net/virtio/virtio_ethdev.h
>> @@ -117,6 +117,8 @@ void virtio_interrupt_handler(void *param);
>>
>> int virtio_dev_pause(struct rte_eth_dev *dev);
>> void virtio_dev_resume(struct rte_eth_dev *dev);
>> +int virtio_dev_stop(struct rte_eth_dev *dev);
>> +int virtio_dev_close(struct rte_eth_dev *dev);
>> int virtio_inject_pkts(struct rte_eth_dev *dev, struct rte_mbuf **tx_pkts,
>> int nb_pkts);
>>
>> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
>> new file mode 100644
>> index 0000000000..9c0935068e
>> --- /dev/null
>> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
>> @@ -0,0 +1,148 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2010-2016 Intel Corporation
>> + */
>> +
>> +#include <stdint.h>
>> +#include <string.h>
>> +#include <stdio.h>
>> +#include <errno.h>
>> +#include <unistd.h>
>> +
>> +#include <rte_ethdev_driver.h>
>> +#include <rte_ethdev_pci.h>
>> +#include <rte_pci.h>
>> +#include <rte_bus_pci.h>
>> +#include <rte_errno.h>
>> +
>> +#include <rte_memory.h>
>> +#include <rte_eal.h>
>> +#include <rte_dev.h>
>> +#include <rte_kvargs.h>
>> +
>> +#include "virtio_ethdev.h"
>> +#include "virtio_pci.h"
>> +#include "virtio_logs.h"
>> +
>> +/*
>> + * The set of PCI devices this driver supports
>> + */
>> +static const struct rte_pci_id pci_id_virtio_map[] = {
>> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_LEGACY_DEVICEID_NET) },
>> + { RTE_PCI_DEVICE(VIRTIO_PCI_VENDORID, VIRTIO_PCI_MODERN_DEVICEID_NET) },
>> + { .vendor_id = 0, /* sentinel */ },
>> +};
>> +
>> +static int
>> +eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
>> +{
>> + return eth_virtio_dev_init(eth_dev);
>> +}
>> +
>> +static int
>> +eth_virtio_pci_uninit(struct rte_eth_dev *eth_dev)
>> +{
>> + int ret;
>> + PMD_INIT_FUNC_TRACE();
>> +
>> + if (rte_eal_process_type() == RTE_PROC_SECONDARY)
>> + return 0;
>> +
>> + ret = virtio_dev_stop(eth_dev);
>> + virtio_dev_close(eth_dev);
>> +
>> + PMD_INIT_LOG(DEBUG, "dev_uninit completed");
>> +
>> + return ret;
>> +}
>> +
>> +static int vdpa_check_handler(__rte_unused const char *key,
>> + const char *value, void *ret_val)
>> +{
>> + if (strcmp(value, "1") == 0)
>> + *(int *)ret_val = 1;
>> + else
>> + *(int *)ret_val = 0;
>> +
>> + return 0;
>> +}
>> +
>> +#define VIRTIO_ARG_VDPA "vdpa"
>> +
>> +static int
>> +virtio_pci_devargs_parse(struct rte_devargs *devargs, int *vdpa)
>> +{
>> + struct rte_kvargs *kvlist;
>> + int ret = 0;
>> +
>> + if (devargs == NULL)
>> + return 0;
>> +
>> + kvlist = rte_kvargs_parse(devargs->args, NULL);
>> + if (kvlist == NULL) {
>> + PMD_INIT_LOG(ERR, "error when parsing param");
>> + return 0;
>> + }
>> +
>> + if (vdpa && rte_kvargs_count(kvlist, VIRTIO_ARG_VDPA) == 1) {
>
> I would remove this check on vdpa pointer being NULL, as it gets
> passed on the stack by a single caller in this file.
I agree this is not necessary.
>
>> + /* vdpa mode selected when there's a key-value pair:
>> + * vdpa=1
>> + */
>> + ret = rte_kvargs_process(kvlist, VIRTIO_ARG_VDPA,
>> + vdpa_check_handler, vdpa);
>> + if (ret < 0)
>> + PMD_INIT_LOG(ERR, "Failed to parse %s", VIRTIO_ARG_VDPA);
>> + }
>> +
>> + rte_kvargs_free(kvlist);
>> +
>> + return ret;
>> +}
>> +
>> +static int eth_virtio_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>> + struct rte_pci_device *pci_dev)
>> +{
>> + int vdpa = 0;
>> + int ret = 0;
>> +
>> + ret = virtio_pci_devargs_parse(pci_dev->device.devargs, &vdpa);
>> + if (ret < 0) {
>> + PMD_INIT_LOG(ERR, "devargs parsing is failed");
>> + return ret;
>> + }
>> + /* virtio pmd skips probe if device needs to work in vdpa mode */
>> + if (vdpa == 1)
>> + return 1;
>> +
>> + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct virtio_pci_dev),
>> + eth_virtio_pci_init);
>> +}
>> +
>> +static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
>> +{
>> + int ret;
>> +
>> + ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_pci_uninit);
>> + /* Port has already been released by close. */
>> + if (ret == -ENODEV)
>> + ret = 0;
>> + return ret;
>> +}
>> +
>> +static struct rte_pci_driver rte_virtio_pmd = {
>
> This seems too generic.
>
> virtio_pci_pmd ?
virtio_net_pci_pmd is even better.
>
>> + .driver = {
>> + .name = "net_virtio",
I keept it as is here not to break tools relying on driver name.
Makes sense?
>> + },
>> + .id_table = pci_id_virtio_map,
>> + .drv_flags = 0,
>> + .probe = eth_virtio_pci_probe,
>> + .remove = eth_virtio_pci_remove,
>> +};
>> +
>> +RTE_INIT(rte_virtio_pmd_init)
>
> virtio_pci_pmd_init ?
>
rte_virtio_net_pci_pmd_init
>
>> +{
>> + rte_eal_iopl_init();
>> + rte_pci_register(&rte_virtio_pmd);
>> +}
>> +
>> +RTE_PMD_REGISTER_PCI_TABLE(net_virtio, pci_id_virtio_map);
>> +RTE_PMD_REGISTER_KMOD_DEP(net_virtio, "* igb_uio | uio_pci_generic | vfio-pci");
>
>
> --
> David Marchand
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file
2021-01-14 16:04 ` Maxime Coquelin
@ 2021-01-14 16:14 ` David Marchand
0 siblings, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-14 16:14 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Thu, Jan 14, 2021 at 5:04 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> On 1/5/21 10:19 PM, David Marchand wrote:
> >> static bool
> >> rx_offload_enabled(struct virtio_hw *hw)
> >> @@ -2521,7 +2419,7 @@ static void virtio_dev_free_mbufs(struct rte_eth_dev *dev)
> >> /*
> >> * Stop device: disable interrupt and mark link down
> >> */
> >> -static int
> >> +int
> >> virtio_dev_stop(struct rte_eth_dev *dev)
> >> {
> >> struct virtio_hw *hw = dev->data->dev_private;
> >> @@ -2673,7 +2571,5 @@ __rte_unused uint8_t is_rx)
> >> }
> >>
> >> RTE_PMD_EXPORT_NAME(net_virtio, __COUNTER__);
> >
> > This belongs with the rest of the pci driver declarations.
> >
> > $ ./usertools/dpdk-pmdinfo.py
> > $HOME/builds/build-gcc-shared/drivers/librte_net_virtio.so
> > PMD NAME: net_virtio_user
> > PMD PARAMETERS: path=<path> mac=<mac addr> cq=<int> queue_size=<int>
> > queues=<int> iface=<string> server=<0|1> mrg_rxbuf=<0|1>
> > in_order=<0|1> packed_vq=<0|1> speed=<int> vectorized=<0|1>
> >
> > PMD NAME: net_virtio
> >
> > ^^^
> > Here, I would expect the pci table and the kmod dependencies.
> >
> >
> > This makes me realise that the net_virtio_user driver is not exported
> > (but it seems to work without it.. interesting).
>
> As discussed offlist, I have moved RTE_PMD_EXPORT_NAME to
> virtio_pci_ethdev.c, and I now get expected output:
Some details, if someone is interested:
In the pci driver case, we can't rely on the pci bus register macro
because of the custom constructor, hence the manual call to
RTE_PMD_EXPORT_NAME.
For the virtio_user driver, the vdev bus register macro actually
exports the driver and this is why it works fine.
>
> $ ./usertools/dpdk-pmdinfo.py build/drivers/librte_net_virtio.so
> PMD NAME: net_virtio_user
> PMD PARAMETERS: path=<path> mac=<mac addr> cq=<int> queue_size=<int>
> queues=<int> iface=<string> server=<0|1> mrg_rxbuf=<0|1> in_order=<0|1>
> packed_vq=<0|1> speed=<int> vectorized=<0|1>
>
> PMD NAME: net_virtio
> PMD KMOD DEPENDENCIES: * igb_uio | uio_pci_generic | vfio-pci
> PMD HW SUPPORT:
> Red Hat, Inc. (1af4) : Virtio network device (1000) (All Subdevices)
> Red Hat, Inc. (1af4) : Virtio network device (1041) (All Subdevices)
[snip]
> >> +static int eth_virtio_pci_remove(struct rte_pci_device *pci_dev)
> >> +{
> >> + int ret;
> >> +
> >> + ret = rte_eth_dev_pci_generic_remove(pci_dev, eth_virtio_pci_uninit);
> >> + /* Port has already been released by close. */
> >> + if (ret == -ENODEV)
> >> + ret = 0;
> >> + return ret;
> >> +}
> >> +
> >> +static struct rte_pci_driver rte_virtio_pmd = {
> >
> > This seems too generic.
> >
> > virtio_pci_pmd ?
>
> virtio_net_pci_pmd is even better.
>
> >
> >> + .driver = {
> >> + .name = "net_virtio",
>
> I keept it as is here not to break tools relying on driver name.
> Makes sense?
Yes.
>
> >> + },
> >> + .id_table = pci_id_virtio_map,
> >> + .drv_flags = 0,
> >> + .probe = eth_virtio_pci_probe,
> >> + .remove = eth_virtio_pci_remove,
> >> +};
> >> +
> >> +RTE_INIT(rte_virtio_pmd_init)
> >
> > virtio_pci_pmd_init ?
> >
>
> rte_virtio_net_pci_pmd_init
Ok for me.
Thanks Maxime.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (4 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 05/40] net/virtio: move PCI device init in dedicated file Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:05 ` Xia, Chenbo
2021-01-06 8:58 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev Maxime Coquelin
` (34 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves the PCI specific initialization from
eth_virtio_dev_init() to eth_virtio_pci_init().
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 63 +----------------------
drivers/net/virtio/virtio_pci_ethdev.c | 71 +++++++++++++++++++++++++-
2 files changed, 71 insertions(+), 63 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index ca21a019e5..4032a89396 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1676,7 +1676,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
struct virtio_hw *hw = eth_dev->data->dev_private;
struct virtio_net_config *config;
struct virtio_net_config local_config;
- struct rte_pci_device *pci_dev = NULL;
int ret;
/* Reset the device although not necessary at startup */
@@ -1697,9 +1696,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
- pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
/* If host does not support both status and MSI-X then disable LSC */
if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS) &&
hw->use_msix != VIRTIO_MSIX_NONE)
@@ -1828,45 +1824,9 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
vtpci_reinit_complete(hw);
- if (pci_dev)
- PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x",
- eth_dev->data->port_id, pci_dev->id.vendor_id,
- pci_dev->id.device_id);
-
return 0;
}
-/*
- * Remap the PCI device again (IO port map for legacy device and
- * memory map for modern device), so that the secondary process
- * could have the PCI initiated correctly.
- */
-static int
-virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
-{
- if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
- /*
- * We don't have to re-parse the PCI config space, since
- * rte_pci_map_device() makes sure the mapped address
- * in secondary process would equal to the one mapped in
- * the primary process: error will be returned if that
- * requirement is not met.
- *
- * That said, we could simply reuse all cap pointers
- * (such as dev_cfg, common_cfg, etc.) parsed from the
- * primary process, which is stored in shared memory.
- */
- if (rte_pci_map_device(pci_dev)) {
- PMD_INIT_LOG(DEBUG, "failed to map pci device!");
- return -1;
- }
- } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
- if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
- return -1;
- }
-
- return 0;
-}
static void
virtio_set_vtpci_ops(struct virtio_hw *hw)
@@ -1906,17 +1866,11 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
- if (hw->bus_type != VIRTIO_BUS_USER) {
- ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
- if (ret)
- return ret;
- }
-
virtio_set_vtpci_ops(hw);
set_rxtx_funcs(eth_dev);
-
return 0;
}
+
ret = virtio_dev_devargs_parse(eth_dev->device->devargs, &speed, &vectorized);
if (ret < 0)
return ret;
@@ -1933,15 +1887,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
}
hw->port_id = eth_dev->data->port_id;
- /* For virtio_user case the hw->virtio_user_dev is populated by
- * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
- */
- if (hw->bus_type != VIRTIO_BUS_USER) {
- ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
- if (ret)
- goto err_vtpci_init;
- }
-
rte_spinlock_init(&hw->state_lock);
/* reset device and negotiate default features */
@@ -1968,12 +1913,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
return 0;
err_virtio_init:
- if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
- rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
- rte_pci_ioport_unmap(VTPCI_IO(hw));
- }
-err_vtpci_init:
rte_free(eth_dev->data->mac_addrs);
eth_dev->data->mac_addrs = NULL;
return ret;
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
index 9c0935068e..d6cbe582d2 100644
--- a/drivers/net/virtio/virtio_pci_ethdev.c
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -32,10 +32,79 @@ static const struct rte_pci_id pci_id_virtio_map[] = {
{ .vendor_id = 0, /* sentinel */ },
};
+
+/*
+ * Remap the PCI device again (IO port map for legacy device and
+ * memory map for modern device), so that the secondary process
+ * could have the PCI initiated correctly.
+ */
+static int
+virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
+{
+ if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
+ /*
+ * We don't have to re-parse the PCI config space, since
+ * rte_pci_map_device() makes sure the mapped address
+ * in secondary process would equal to the one mapped in
+ * the primary process: error will be returned if that
+ * requirement is not met.
+ *
+ * That said, we could simply reuse all cap pointers
+ * (such as dev_cfg, common_cfg, etc.) parsed from the
+ * primary process, which is stored in shared memory.
+ */
+ if (rte_pci_map_device(pci_dev)) {
+ PMD_INIT_LOG(DEBUG, "failed to map pci device!");
+ return -1;
+ }
+ } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
+ if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
+ return -1;
+ }
+
+ return 0;
+}
+
static int
eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
{
- return eth_virtio_dev_init(eth_dev);
+ struct virtio_pci_dev *dev = eth_dev->data->dev_private;
+ struct virtio_hw *hw = &dev->hw;
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+ int ret;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init PCI device\n");
+ return -1;
+ }
+ } else {
+ ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
+ return -1;
+ }
+ }
+
+ ret = eth_virtio_dev_init(eth_dev);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to init virtio device\n");
+ goto err_unmap;
+ }
+
+ PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x",
+ eth_dev->data->port_id, pci_dev->id.vendor_id,
+ pci_dev->id.device_id);
+
+ return 0;
+
+err_unmap:
+ rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
+ if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
+ rte_pci_ioport_unmap(VTPCI_IO(hw));
+
+ return ret;
}
static int
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init
2020-12-20 21:13 ` [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init Maxime Coquelin
@ 2020-12-30 3:05 ` Xia, Chenbo
2021-01-06 8:58 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:05 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev
> init
>
> This patch moves the PCI specific initialization from
> eth_virtio_dev_init() to eth_virtio_pci_init().
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 63 +----------------------
> drivers/net/virtio/virtio_pci_ethdev.c | 71 +++++++++++++++++++++++++-
> 2 files changed, 71 insertions(+), 63 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index ca21a019e5..4032a89396 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1676,7 +1676,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> struct virtio_hw *hw = eth_dev->data->dev_private;
> struct virtio_net_config *config;
> struct virtio_net_config local_config;
> - struct rte_pci_device *pci_dev = NULL;
> int ret;
>
> /* Reset the device although not necessary at startup */
> @@ -1697,9 +1696,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
>
> hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> - pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
> -
> /* If host does not support both status and MSI-X then disable LSC */
> if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS) &&
> hw->use_msix != VIRTIO_MSIX_NONE)
> @@ -1828,45 +1824,9 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
>
> vtpci_reinit_complete(hw);
>
> - if (pci_dev)
> - PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x",
> - eth_dev->data->port_id, pci_dev->id.vendor_id,
> - pci_dev->id.device_id);
> -
> return 0;
> }
>
> -/*
> - * Remap the PCI device again (IO port map for legacy device and
> - * memory map for modern device), so that the secondary process
> - * could have the PCI initiated correctly.
> - */
> -static int
> -virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> -{
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
> - /*
> - * We don't have to re-parse the PCI config space, since
> - * rte_pci_map_device() makes sure the mapped address
> - * in secondary process would equal to the one mapped in
> - * the primary process: error will be returned if that
> - * requirement is not met.
> - *
> - * That said, we could simply reuse all cap pointers
> - * (such as dev_cfg, common_cfg, etc.) parsed from the
> - * primary process, which is stored in shared memory.
> - */
> - if (rte_pci_map_device(pci_dev)) {
> - PMD_INIT_LOG(DEBUG, "failed to map pci device!");
> - return -1;
> - }
> - } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
> - if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
> - return -1;
> - }
> -
> - return 0;
> -}
>
> static void
> virtio_set_vtpci_ops(struct virtio_hw *hw)
> @@ -1906,17 +1866,11 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> - if (hw->bus_type != VIRTIO_BUS_USER) {
> - ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> - if (ret)
> - return ret;
> - }
> -
> virtio_set_vtpci_ops(hw);
> set_rxtx_funcs(eth_dev);
> -
> return 0;
> }
> +
> ret = virtio_dev_devargs_parse(eth_dev->device->devargs, &speed,
> &vectorized);
> if (ret < 0)
> return ret;
> @@ -1933,15 +1887,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> }
>
> hw->port_id = eth_dev->data->port_id;
> - /* For virtio_user case the hw->virtio_user_dev is populated by
> - * virtio_user_eth_dev_alloc() before eth_virtio_dev_init() is called.
> - */
> - if (hw->bus_type != VIRTIO_BUS_USER) {
> - ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> - if (ret)
> - goto err_vtpci_init;
> - }
> -
> rte_spinlock_init(&hw->state_lock);
>
> /* reset device and negotiate default features */
> @@ -1968,12 +1913,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> return 0;
>
> err_virtio_init:
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type ==
> VIRTIO_BUS_PCI_LEGACY) {
> - rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> - rte_pci_ioport_unmap(VTPCI_IO(hw));
> - }
> -err_vtpci_init:
> rte_free(eth_dev->data->mac_addrs);
> eth_dev->data->mac_addrs = NULL;
> return ret;
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> index 9c0935068e..d6cbe582d2 100644
> --- a/drivers/net/virtio/virtio_pci_ethdev.c
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -32,10 +32,79 @@ static const struct rte_pci_id pci_id_virtio_map[] = {
> { .vendor_id = 0, /* sentinel */ },
> };
>
> +
> +/*
> + * Remap the PCI device again (IO port map for legacy device and
> + * memory map for modern device), so that the secondary process
> + * could have the PCI initiated correctly.
> + */
> +static int
> +virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> +{
> + if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
> + /*
> + * We don't have to re-parse the PCI config space, since
> + * rte_pci_map_device() makes sure the mapped address
> + * in secondary process would equal to the one mapped in
> + * the primary process: error will be returned if that
> + * requirement is not met.
> + *
> + * That said, we could simply reuse all cap pointers
> + * (such as dev_cfg, common_cfg, etc.) parsed from the
> + * primary process, which is stored in shared memory.
> + */
> + if (rte_pci_map_device(pci_dev)) {
> + PMD_INIT_LOG(DEBUG, "failed to map pci device!");
> + return -1;
> + }
> + } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
> + if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> static int
> eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> {
> - return eth_virtio_dev_init(eth_dev);
> + struct virtio_pci_dev *dev = eth_dev->data->dev_private;
> + struct virtio_hw *hw = &dev->hw;
> + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
> + int ret;
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init PCI device\n");
> + return -1;
> + }
> + } else {
> + ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
> + return -1;
> + }
> + }
> +
> + ret = eth_virtio_dev_init(eth_dev);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "Failed to init virtio device\n");
> + goto err_unmap;
> + }
> +
> + PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x",
> + eth_dev->data->port_id, pci_dev->id.vendor_id,
> + pci_dev->id.device_id);
> +
> + return 0;
> +
> +err_unmap:
> + rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> + if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> + rte_pci_ioport_unmap(VTPCI_IO(hw));
> +
> + return ret;
> }
>
> static int
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init
2020-12-20 21:13 ` [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init Maxime Coquelin
2020-12-30 3:05 ` Xia, Chenbo
@ 2021-01-06 8:58 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 8:58 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves the PCI specific initialization from
> eth_virtio_dev_init() to eth_virtio_pci_init().
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (5 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 06/40] net/virtio: move PCI specific dev init to PCI ethdev init Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:05 ` Xia, Chenbo
2021-01-06 8:22 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user Maxime Coquelin
` (33 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
There is no point it detecting whether we can use MSIX
every time the interrupt is enabled/disabled/masked.
Let's do it once for all at PCI device init time.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 15 ---------------
drivers/net/virtio/virtio_pci.c | 5 ++++-
2 files changed, 4 insertions(+), 16 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 4032a89396..67f6be3fa8 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1282,42 +1282,27 @@ virtio_vlan_filter_set(struct rte_eth_dev *dev, uint16_t vlan_id, int on)
static int
virtio_intr_unmask(struct rte_eth_dev *dev)
{
- struct virtio_hw *hw = dev->data->dev_private;
-
if (rte_intr_ack(dev->intr_handle) < 0)
return -1;
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
- hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
-
return 0;
}
static int
virtio_intr_enable(struct rte_eth_dev *dev)
{
- struct virtio_hw *hw = dev->data->dev_private;
-
if (rte_intr_enable(dev->intr_handle) < 0)
return -1;
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
- hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
-
return 0;
}
static int
virtio_intr_disable(struct rte_eth_dev *dev)
{
- struct virtio_hw *hw = dev->data->dev_private;
-
if (rte_intr_disable(dev->intr_handle) < 0)
return -1;
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN)
- hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
-
return 0;
}
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 1692268f30..8605254e53 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -698,7 +698,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
PMD_INIT_LOG(INFO, "modern virtio pci detected.");
virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
hw->bus_type = VIRTIO_BUS_PCI_MODERN;
- return 0;
+ goto msix_detect;
}
PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
@@ -718,6 +718,9 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
+msix_detect:
+ hw->use_msix = vtpci_msix_detect(dev);
+
return 0;
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev
2020-12-20 21:13 ` [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev Maxime Coquelin
@ 2020-12-30 3:05 ` Xia, Chenbo
2021-01-06 8:22 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:05 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev
>
> There is no point it detecting whether we can use MSIX
> every time the interrupt is enabled/disabled/masked.
>
> Let's do it once for all at PCI device init time.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 15 ---------------
> drivers/net/virtio/virtio_pci.c | 5 ++++-
> 2 files changed, 4 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 4032a89396..67f6be3fa8 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1282,42 +1282,27 @@ virtio_vlan_filter_set(struct rte_eth_dev *dev,
> uint16_t vlan_id, int on)
> static int
> virtio_intr_unmask(struct rte_eth_dev *dev)
> {
> - struct virtio_hw *hw = dev->data->dev_private;
> -
> if (rte_intr_ack(dev->intr_handle) < 0)
> return -1;
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> - hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
> -
> return 0;
> }
>
> static int
> virtio_intr_enable(struct rte_eth_dev *dev)
> {
> - struct virtio_hw *hw = dev->data->dev_private;
> -
> if (rte_intr_enable(dev->intr_handle) < 0)
> return -1;
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> - hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
> -
> return 0;
> }
>
> static int
> virtio_intr_disable(struct rte_eth_dev *dev)
> {
> - struct virtio_hw *hw = dev->data->dev_private;
> -
> if (rte_intr_disable(dev->intr_handle) < 0)
> return -1;
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN)
> - hw->use_msix = vtpci_msix_detect(RTE_ETH_DEV_TO_PCI(dev));
> -
> return 0;
> }
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 1692268f30..8605254e53 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -698,7 +698,7 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw
> *hw)
> PMD_INIT_LOG(INFO, "modern virtio pci detected.");
> virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
> hw->bus_type = VIRTIO_BUS_PCI_MODERN;
> - return 0;
> + goto msix_detect;
> }
>
> PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
> @@ -718,6 +718,9 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw
> *hw)
> virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
> hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
>
> +msix_detect:
> + hw->use_msix = vtpci_msix_detect(dev);
> +
> return 0;
> }
>
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev
2020-12-20 21:13 ` [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev Maxime Coquelin
2020-12-30 3:05 ` Xia, Chenbo
@ 2021-01-06 8:22 ` David Marchand
2021-01-14 17:19 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 8:22 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> There is no point it detecting whether we can use MSIX
> every time the interrupt is enabled/disabled/masked.
>
> Let's do it once for all at PCI device init time.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Is this a rework/fix of fe19d49cb525 ("net/virtio: fix Rx interrupt
with VFIO") ?
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev
2021-01-06 8:22 ` David Marchand
@ 2021-01-14 17:19 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 17:19 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 9:22 AM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> There is no point it detecting whether we can use MSIX
>> every time the interrupt is enabled/disabled/masked.
>>
>> Let's do it once for all at PCI device init time.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
> Is this a rework/fix of fe19d49cb525 ("net/virtio: fix Rx interrupt
> with VFIO") ?
Hmm, that was only planned as a rework, but it is likely to introduce a
regression by reverting the fix you mention.
I think I'll have to introduce a new callback so that drivers can update
their status.
Good catch, thanks!
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (6 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 07/40] net/virtio: move MSIX detection to PCI ethdev Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:06 ` Xia, Chenbo
2021-01-06 9:06 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata Maxime Coquelin
` (32 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
At least Vhost-user backend of Virtio-user PMD requires
IOVA as VA mode. Until now, it was implemented as a hack
by forcing to use mbuf's buf_addr field instead of buf_iova.
This patcv removes all this logic and just fails probing
if IOVA as VA mode is not selected. It simplifies the
code overall, and removes some bus-specific logic from
generic virtio_ethdev.c.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 15 ---------
drivers/net/virtio/virtio_rxtx.c | 34 +++++++++------------
drivers/net/virtio/virtio_rxtx_packed_avx.c | 10 +++---
drivers/net/virtio/virtio_rxtx_simple.h | 3 +-
drivers/net/virtio/virtio_user_ethdev.c | 11 +++++++
drivers/net/virtio/virtqueue.h | 25 +--------------
6 files changed, 32 insertions(+), 66 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 67f6be3fa8..13e2ec998a 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -576,21 +576,6 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
hw->cvq = cvq;
}
- /* For virtio_user case (that is when hw->virtio_user_dev is not NULL),
- * we use virtual address. And we need properly set _offset_, please see
- * VIRTIO_MBUF_DATA_DMA_ADDR in virtqueue.h for more information.
- */
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
- vq->offset = offsetof(struct rte_mbuf, buf_iova);
- } else if (hw->bus_type == VIRTIO_BUS_USER) {
- vq->vq_ring_mem = (uintptr_t)mz->addr;
- vq->offset = offsetof(struct rte_mbuf, buf_addr);
- if (queue_type == VTNET_TQ)
- txvq->virtio_net_hdr_mem = (uintptr_t)hdr_mz->addr;
- else if (queue_type == VTNET_CQ)
- cvq->virtio_net_hdr_mem = (uintptr_t)hdr_mz->addr;
- }
-
if (queue_type == VTNET_TQ) {
struct virtio_tx_region *txr;
unsigned int i;
diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index 77934e8c58..93fe856cbd 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -271,13 +271,10 @@ virtqueue_enqueue_refill_inorder(struct virtqueue *vq,
dxp->cookie = (void *)cookies[i];
dxp->ndescs = 1;
- start_dp[idx].addr =
- VIRTIO_MBUF_ADDR(cookies[i], vq) +
- RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
- start_dp[idx].len =
- cookies[i]->buf_len -
- RTE_PKTMBUF_HEADROOM +
- hw->vtnet_hdr_size;
+ start_dp[idx].addr = cookies[i]->buf_iova +
+ RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
+ start_dp[idx].len = cookies[i]->buf_len -
+ RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
start_dp[idx].flags = VRING_DESC_F_WRITE;
vq_update_avail_ring(vq, idx);
@@ -313,12 +310,10 @@ virtqueue_enqueue_recv_refill(struct virtqueue *vq, struct rte_mbuf **cookie,
dxp->cookie = (void *)cookie[i];
dxp->ndescs = 1;
- start_dp[idx].addr =
- VIRTIO_MBUF_ADDR(cookie[i], vq) +
+ start_dp[idx].addr = cookie[i]->buf_iova +
RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
- start_dp[idx].len =
- cookie[i]->buf_len - RTE_PKTMBUF_HEADROOM +
- hw->vtnet_hdr_size;
+ start_dp[idx].len = cookie[i]->buf_len -
+ RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
start_dp[idx].flags = VRING_DESC_F_WRITE;
vq->vq_desc_head_idx = start_dp[idx].next;
vq_update_avail_ring(vq, idx);
@@ -355,10 +350,10 @@ virtqueue_enqueue_recv_refill_packed(struct virtqueue *vq,
dxp->cookie = (void *)cookie[i];
dxp->ndescs = 1;
- start_dp[idx].addr = VIRTIO_MBUF_ADDR(cookie[i], vq) +
- RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
- start_dp[idx].len = cookie[i]->buf_len - RTE_PKTMBUF_HEADROOM
- + hw->vtnet_hdr_size;
+ start_dp[idx].addr = cookie[i]->buf_iova +
+ RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
+ start_dp[idx].len = cookie[i]->buf_len -
+ RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
vq->vq_desc_head_idx = dxp->next;
if (vq->vq_desc_head_idx == VQ_RING_DESC_CHAIN_END)
@@ -455,8 +450,7 @@ virtqueue_enqueue_xmit_inorder(struct virtnet_tx *txvq,
else
virtqueue_xmit_offload(hdr, cookies[i], true);
- start_dp[idx].addr =
- VIRTIO_MBUF_DATA_DMA_ADDR(cookies[i], vq) - head_size;
+ start_dp[idx].addr = rte_mbuf_data_iova(cookies[i]) - head_size;
start_dp[idx].len = cookies[i]->data_len + head_size;
start_dp[idx].flags = 0;
@@ -503,7 +497,7 @@ virtqueue_enqueue_xmit_packed_fast(struct virtnet_tx *txvq,
else
virtqueue_xmit_offload(hdr, cookie, true);
- dp->addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq) - head_size;
+ dp->addr = rte_mbuf_data_iova(cookie) - head_size;
dp->len = cookie->data_len + head_size;
dp->id = id;
@@ -590,7 +584,7 @@ virtqueue_enqueue_xmit(struct virtnet_tx *txvq, struct rte_mbuf *cookie,
virtqueue_xmit_offload(hdr, cookie, vq->hw->has_tx_offload);
do {
- start_dp[idx].addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq);
+ start_dp[idx].addr = rte_mbuf_data_iova(cookie);
start_dp[idx].len = cookie->data_len;
if (prepend_header) {
start_dp[idx].addr -= head_size;
diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c b/drivers/net/virtio/virtio_rxtx_packed_avx.c
index 9bc62719ee..a6a49ec439 100644
--- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
+++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
@@ -133,13 +133,13 @@ virtqueue_enqueue_batch_packed_vec(struct virtnet_tx *txvq,
}
__m512i descs_base = _mm512_set_epi64(tx_pkts[3]->data_len,
- VIRTIO_MBUF_ADDR(tx_pkts[3], vq),
+ tx_pkts[3]->buf_iova,
tx_pkts[2]->data_len,
- VIRTIO_MBUF_ADDR(tx_pkts[2], vq),
+ tx_pkts[2]->buf_iova,
tx_pkts[1]->data_len,
- VIRTIO_MBUF_ADDR(tx_pkts[1], vq),
+ tx_pkts[1]->buf_iova,
tx_pkts[0]->data_len,
- VIRTIO_MBUF_ADDR(tx_pkts[0], vq));
+ tx_pkts[0]->buf_iova);
/* id offset and data offset */
__m512i data_offsets = _mm512_set_epi64((uint64_t)3 << ID_BITS_OFFSET,
@@ -536,7 +536,7 @@ virtio_recv_refill_packed_vec(struct virtnet_rx *rxvq,
dxp = &vq->vq_descx[idx + i];
dxp->cookie = (void *)cookie[total_num + i];
- addr = VIRTIO_MBUF_ADDR(cookie[total_num + i], vq) +
+ addr = cookie[total_num + i]->buf_iova +
RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
start_dp[idx + i].addr = addr;
start_dp[idx + i].len = cookie[total_num + i]->buf_len
diff --git a/drivers/net/virtio/virtio_rxtx_simple.h b/drivers/net/virtio/virtio_rxtx_simple.h
index 3d1296a23c..f2a5aedf97 100644
--- a/drivers/net/virtio/virtio_rxtx_simple.h
+++ b/drivers/net/virtio/virtio_rxtx_simple.h
@@ -43,8 +43,7 @@ virtio_rxq_rearm_vec(struct virtnet_rx *rxvq)
p = (uintptr_t)&sw_ring[i]->rearm_data;
*(uint64_t *)p = rxvq->mbuf_initializer;
- start_dp[i].addr =
- VIRTIO_MBUF_ADDR(sw_ring[i], vq) +
+ start_dp[i].addr = sw_ring[i]->buf_iova +
RTE_PKTMBUF_HEADROOM - vq->hw->vtnet_hdr_size;
start_dp[i].len = sw_ring[i]->buf_len -
RTE_PKTMBUF_HEADROOM + vq->hw->vtnet_hdr_size;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 1f1f63a1a5..f4775ff141 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
char *mac_addr = NULL;
int ret = -1;
+ /*
+ * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
+ * it implies API breakage.
+ * ToDo 2: Check if all backends have this requirement. Likely
+ * Vhost-vDPA and Vhost-Kernel are fine with PA IOVA mode.
+ */
+ if (rte_eal_iova_mode() != RTE_IOVA_VA) {
+ PMD_INIT_LOG(ERR, "Probing failed, only VA IOVA mode supported\n");
+ return -1;
+ }
+
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
const char *name = rte_vdev_device_name(vdev);
eth_dev = rte_eth_dev_attach_secondary(name);
diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
index 42c4c9882f..e4a1393816 100644
--- a/drivers/net/virtio/virtqueue.h
+++ b/drivers/net/virtio/virtqueue.h
@@ -114,29 +114,6 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp,
#define VIRTQUEUE_MAX_NAME_SZ 32
-#ifdef RTE_VIRTIO_USER
-/**
- * Return the physical address (or virtual address in case of
- * virtio-user) of mbuf data buffer.
- *
- * The address is firstly casted to the word size (sizeof(uintptr_t))
- * before casting it to uint64_t. This is to make it work with different
- * combination of word size (64 bit and 32 bit) and virtio device
- * (virtio-pci and virtio-user).
- */
-#define VIRTIO_MBUF_ADDR(mb, vq) \
- ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->offset)))
-#else
-#define VIRTIO_MBUF_ADDR(mb, vq) ((mb)->buf_iova)
-#endif
-
-/**
- * Return the physical address (or virtual address in case of
- * virtio-user) of mbuf data buffer, taking care of mbuf data offset
- */
-#define VIRTIO_MBUF_DATA_DMA_ADDR(mb, vq) \
- (VIRTIO_MBUF_ADDR(mb, vq) + (mb)->data_off)
-
#define VTNET_SQ_RQ_QUEUE_IDX 0
#define VTNET_SQ_TQ_QUEUE_IDX 1
#define VTNET_SQ_CQ_QUEUE_IDX 2
@@ -764,7 +741,7 @@ virtqueue_enqueue_xmit_packed(struct virtnet_tx *txvq, struct rte_mbuf *cookie,
do {
uint16_t flags;
- start_dp[idx].addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq);
+ start_dp[idx].addr = rte_mbuf_data_iova(cookie);
start_dp[idx].len = cookie->data_len;
if (prepend_header) {
start_dp[idx].addr -= head_size;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2020-12-20 21:13 ` [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user Maxime Coquelin
@ 2020-12-30 3:06 ` Xia, Chenbo
2021-01-06 9:06 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:06 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
>
> At least Vhost-user backend of Virtio-user PMD requires
> IOVA as VA mode. Until now, it was implemented as a hack
> by forcing to use mbuf's buf_addr field instead of buf_iova.
>
> This patcv removes all this logic and just fails probing
s/patcv/patch
With this fix:
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
> if IOVA as VA mode is not selected. It simplifies the
> code overall, and removes some bus-specific logic from
> generic virtio_ethdev.c.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 15 ---------
> drivers/net/virtio/virtio_rxtx.c | 34 +++++++++------------
> drivers/net/virtio/virtio_rxtx_packed_avx.c | 10 +++---
> drivers/net/virtio/virtio_rxtx_simple.h | 3 +-
> drivers/net/virtio/virtio_user_ethdev.c | 11 +++++++
> drivers/net/virtio/virtqueue.h | 25 +--------------
> 6 files changed, 32 insertions(+), 66 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 67f6be3fa8..13e2ec998a 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -576,21 +576,6 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> hw->cvq = cvq;
> }
>
> - /* For virtio_user case (that is when hw->virtio_user_dev is not NULL),
> - * we use virtual address. And we need properly set _offset_, please see
> - * VIRTIO_MBUF_DATA_DMA_ADDR in virtqueue.h for more information.
> - */
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY || hw->bus_type ==
> VIRTIO_BUS_PCI_MODERN) {
> - vq->offset = offsetof(struct rte_mbuf, buf_iova);
> - } else if (hw->bus_type == VIRTIO_BUS_USER) {
> - vq->vq_ring_mem = (uintptr_t)mz->addr;
> - vq->offset = offsetof(struct rte_mbuf, buf_addr);
> - if (queue_type == VTNET_TQ)
> - txvq->virtio_net_hdr_mem = (uintptr_t)hdr_mz->addr;
> - else if (queue_type == VTNET_CQ)
> - cvq->virtio_net_hdr_mem = (uintptr_t)hdr_mz->addr;
> - }
> -
> if (queue_type == VTNET_TQ) {
> struct virtio_tx_region *txr;
> unsigned int i;
> diff --git a/drivers/net/virtio/virtio_rxtx.c
> b/drivers/net/virtio/virtio_rxtx.c
> index 77934e8c58..93fe856cbd 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -271,13 +271,10 @@ virtqueue_enqueue_refill_inorder(struct virtqueue *vq,
> dxp->cookie = (void *)cookies[i];
> dxp->ndescs = 1;
>
> - start_dp[idx].addr =
> - VIRTIO_MBUF_ADDR(cookies[i], vq) +
> - RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> - start_dp[idx].len =
> - cookies[i]->buf_len -
> - RTE_PKTMBUF_HEADROOM +
> - hw->vtnet_hdr_size;
> + start_dp[idx].addr = cookies[i]->buf_iova +
> + RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> + start_dp[idx].len = cookies[i]->buf_len -
> + RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
> start_dp[idx].flags = VRING_DESC_F_WRITE;
>
> vq_update_avail_ring(vq, idx);
> @@ -313,12 +310,10 @@ virtqueue_enqueue_recv_refill(struct virtqueue *vq,
> struct rte_mbuf **cookie,
> dxp->cookie = (void *)cookie[i];
> dxp->ndescs = 1;
>
> - start_dp[idx].addr =
> - VIRTIO_MBUF_ADDR(cookie[i], vq) +
> + start_dp[idx].addr = cookie[i]->buf_iova +
> RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> - start_dp[idx].len =
> - cookie[i]->buf_len - RTE_PKTMBUF_HEADROOM +
> - hw->vtnet_hdr_size;
> + start_dp[idx].len = cookie[i]->buf_len -
> + RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
> start_dp[idx].flags = VRING_DESC_F_WRITE;
> vq->vq_desc_head_idx = start_dp[idx].next;
> vq_update_avail_ring(vq, idx);
> @@ -355,10 +350,10 @@ virtqueue_enqueue_recv_refill_packed(struct virtqueue
> *vq,
> dxp->cookie = (void *)cookie[i];
> dxp->ndescs = 1;
>
> - start_dp[idx].addr = VIRTIO_MBUF_ADDR(cookie[i], vq) +
> - RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> - start_dp[idx].len = cookie[i]->buf_len - RTE_PKTMBUF_HEADROOM
> - + hw->vtnet_hdr_size;
> + start_dp[idx].addr = cookie[i]->buf_iova +
> + RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> + start_dp[idx].len = cookie[i]->buf_len -
> + RTE_PKTMBUF_HEADROOM + hw->vtnet_hdr_size;
>
> vq->vq_desc_head_idx = dxp->next;
> if (vq->vq_desc_head_idx == VQ_RING_DESC_CHAIN_END)
> @@ -455,8 +450,7 @@ virtqueue_enqueue_xmit_inorder(struct virtnet_tx *txvq,
> else
> virtqueue_xmit_offload(hdr, cookies[i], true);
>
> - start_dp[idx].addr =
> - VIRTIO_MBUF_DATA_DMA_ADDR(cookies[i], vq) - head_size;
> + start_dp[idx].addr = rte_mbuf_data_iova(cookies[i]) - head_size;
> start_dp[idx].len = cookies[i]->data_len + head_size;
> start_dp[idx].flags = 0;
>
> @@ -503,7 +497,7 @@ virtqueue_enqueue_xmit_packed_fast(struct virtnet_tx *txvq,
> else
> virtqueue_xmit_offload(hdr, cookie, true);
>
> - dp->addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq) - head_size;
> + dp->addr = rte_mbuf_data_iova(cookie) - head_size;
> dp->len = cookie->data_len + head_size;
> dp->id = id;
>
> @@ -590,7 +584,7 @@ virtqueue_enqueue_xmit(struct virtnet_tx *txvq, struct
> rte_mbuf *cookie,
> virtqueue_xmit_offload(hdr, cookie, vq->hw->has_tx_offload);
>
> do {
> - start_dp[idx].addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq);
> + start_dp[idx].addr = rte_mbuf_data_iova(cookie);
> start_dp[idx].len = cookie->data_len;
> if (prepend_header) {
> start_dp[idx].addr -= head_size;
> diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> index 9bc62719ee..a6a49ec439 100644
> --- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> @@ -133,13 +133,13 @@ virtqueue_enqueue_batch_packed_vec(struct virtnet_tx
> *txvq,
> }
>
> __m512i descs_base = _mm512_set_epi64(tx_pkts[3]->data_len,
> - VIRTIO_MBUF_ADDR(tx_pkts[3], vq),
> + tx_pkts[3]->buf_iova,
> tx_pkts[2]->data_len,
> - VIRTIO_MBUF_ADDR(tx_pkts[2], vq),
> + tx_pkts[2]->buf_iova,
> tx_pkts[1]->data_len,
> - VIRTIO_MBUF_ADDR(tx_pkts[1], vq),
> + tx_pkts[1]->buf_iova,
> tx_pkts[0]->data_len,
> - VIRTIO_MBUF_ADDR(tx_pkts[0], vq));
> + tx_pkts[0]->buf_iova);
>
> /* id offset and data offset */
> __m512i data_offsets = _mm512_set_epi64((uint64_t)3 << ID_BITS_OFFSET,
> @@ -536,7 +536,7 @@ virtio_recv_refill_packed_vec(struct virtnet_rx *rxvq,
> dxp = &vq->vq_descx[idx + i];
> dxp->cookie = (void *)cookie[total_num + i];
>
> - addr = VIRTIO_MBUF_ADDR(cookie[total_num + i], vq) +
> + addr = cookie[total_num + i]->buf_iova +
> RTE_PKTMBUF_HEADROOM - hw->vtnet_hdr_size;
> start_dp[idx + i].addr = addr;
> start_dp[idx + i].len = cookie[total_num + i]->buf_len
> diff --git a/drivers/net/virtio/virtio_rxtx_simple.h
> b/drivers/net/virtio/virtio_rxtx_simple.h
> index 3d1296a23c..f2a5aedf97 100644
> --- a/drivers/net/virtio/virtio_rxtx_simple.h
> +++ b/drivers/net/virtio/virtio_rxtx_simple.h
> @@ -43,8 +43,7 @@ virtio_rxq_rearm_vec(struct virtnet_rx *rxvq)
> p = (uintptr_t)&sw_ring[i]->rearm_data;
> *(uint64_t *)p = rxvq->mbuf_initializer;
>
> - start_dp[i].addr =
> - VIRTIO_MBUF_ADDR(sw_ring[i], vq) +
> + start_dp[i].addr = sw_ring[i]->buf_iova +
> RTE_PKTMBUF_HEADROOM - vq->hw->vtnet_hdr_size;
> start_dp[i].len = sw_ring[i]->buf_len -
> RTE_PKTMBUF_HEADROOM + vq->hw->vtnet_hdr_size;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 1f1f63a1a5..f4775ff141 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
> char *mac_addr = NULL;
> int ret = -1;
>
> + /*
> + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
> + * it implies API breakage.
> + * ToDo 2: Check if all backends have this requirement. Likely
> + * Vhost-vDPA and Vhost-Kernel are fine with PA IOVA mode.
> + */
> + if (rte_eal_iova_mode() != RTE_IOVA_VA) {
> + PMD_INIT_LOG(ERR, "Probing failed, only VA IOVA mode supported\n");
> + return -1;
> + }
> +
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> const char *name = rte_vdev_device_name(vdev);
> eth_dev = rte_eth_dev_attach_secondary(name);
> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
> index 42c4c9882f..e4a1393816 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -114,29 +114,6 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp,
>
> #define VIRTQUEUE_MAX_NAME_SZ 32
>
> -#ifdef RTE_VIRTIO_USER
> -/**
> - * Return the physical address (or virtual address in case of
> - * virtio-user) of mbuf data buffer.
> - *
> - * The address is firstly casted to the word size (sizeof(uintptr_t))
> - * before casting it to uint64_t. This is to make it work with different
> - * combination of word size (64 bit and 32 bit) and virtio device
> - * (virtio-pci and virtio-user).
> - */
> -#define VIRTIO_MBUF_ADDR(mb, vq) \
> - ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->offset)))
> -#else
> -#define VIRTIO_MBUF_ADDR(mb, vq) ((mb)->buf_iova)
> -#endif
> -
> -/**
> - * Return the physical address (or virtual address in case of
> - * virtio-user) of mbuf data buffer, taking care of mbuf data offset
> - */
> -#define VIRTIO_MBUF_DATA_DMA_ADDR(mb, vq) \
> - (VIRTIO_MBUF_ADDR(mb, vq) + (mb)->data_off)
> -
> #define VTNET_SQ_RQ_QUEUE_IDX 0
> #define VTNET_SQ_TQ_QUEUE_IDX 1
> #define VTNET_SQ_CQ_QUEUE_IDX 2
> @@ -764,7 +741,7 @@ virtqueue_enqueue_xmit_packed(struct virtnet_tx *txvq,
> struct rte_mbuf *cookie,
> do {
> uint16_t flags;
>
> - start_dp[idx].addr = VIRTIO_MBUF_DATA_DMA_ADDR(cookie, vq);
> + start_dp[idx].addr = rte_mbuf_data_iova(cookie);
> start_dp[idx].len = cookie->data_len;
> if (prepend_header) {
> start_dp[idx].addr -= head_size;
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2020-12-20 21:13 ` [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user Maxime Coquelin
2020-12-30 3:06 ` Xia, Chenbo
@ 2021-01-06 9:06 ` David Marchand
2021-01-06 9:11 ` Thomas Monjalon
2021-01-06 9:14 ` Maxime Coquelin
1 sibling, 2 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:06 UTC (permalink / raw)
To: Maxime Coquelin, Ray Kinsella, Thomas Monjalon
Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> index 1f1f63a1a5..f4775ff141 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
> char *mac_addr = NULL;
> int ret = -1;
>
> + /*
> + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
> + * it implies API breakage.
Extending rte_vdev_driver to implement this detection would be an ABI breakage.
This is a driver-only API (rte_vdev_driver is only used by the vdev
bus and drivers afaics).
Doing this is allowed as per my understanding of the ABI policy which
guarantees ABI stability for applications.
We do not guarantee this stability for OOT drivers.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2021-01-06 9:06 ` David Marchand
@ 2021-01-06 9:11 ` Thomas Monjalon
2021-01-06 9:22 ` Maxime Coquelin
2021-01-06 16:37 ` Kinsella, Ray
2021-01-06 9:14 ` Maxime Coquelin
1 sibling, 2 replies; 149+ messages in thread
From: Thomas Monjalon @ 2021-01-06 9:11 UTC (permalink / raw)
To: Maxime Coquelin, David Marchand
Cc: Ray Kinsella, dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
06/01/2021 10:06, David Marchand:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
> > diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> > index 1f1f63a1a5..f4775ff141 100644
> > --- a/drivers/net/virtio/virtio_user_ethdev.c
> > +++ b/drivers/net/virtio/virtio_user_ethdev.c
> > @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
> > char *mac_addr = NULL;
> > int ret = -1;
> >
> > + /*
> > + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
> > + * it implies API breakage.
>
> Extending rte_vdev_driver to implement this detection would be an ABI breakage.
> This is a driver-only API (rte_vdev_driver is only used by the vdev
> bus and drivers afaics).
>
> Doing this is allowed as per my understanding of the ABI policy which
> guarantees ABI stability for applications.
> We do not guarantee this stability for OOT drivers.
I agree.
As a reminder, the A in ABI stands for Application.
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2021-01-06 9:11 ` Thomas Monjalon
@ 2021-01-06 9:22 ` Maxime Coquelin
2021-01-06 16:37 ` Kinsella, Ray
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-06 9:22 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand
Cc: Ray Kinsella, dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 10:11 AM, Thomas Monjalon wrote:
> 06/01/2021 10:06, David Marchand:
>> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
>> <maxime.coquelin@redhat.com> wrote:
>>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
>>> index 1f1f63a1a5..f4775ff141 100644
>>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>>> @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
>>> char *mac_addr = NULL;
>>> int ret = -1;
>>>
>>> + /*
>>> + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
>>> + * it implies API breakage.
>>
>> Extending rte_vdev_driver to implement this detection would be an ABI breakage.
>> This is a driver-only API (rte_vdev_driver is only used by the vdev
>> bus and drivers afaics).
>>
>> Doing this is allowed as per my understanding of the ABI policy which
>> guarantees ABI stability for applications.
>> We do not guarantee this stability for OOT drivers.
>
> I agree.
> As a reminder, the A in ABI stands for Application.
Cool, so we're all good.
Thanks for the prompt reply!
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2021-01-06 9:11 ` Thomas Monjalon
2021-01-06 9:22 ` Maxime Coquelin
@ 2021-01-06 16:37 ` Kinsella, Ray
1 sibling, 0 replies; 149+ messages in thread
From: Kinsella, Ray @ 2021-01-06 16:37 UTC (permalink / raw)
To: Thomas Monjalon, Maxime Coquelin, David Marchand
Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 06/01/2021 09:11, Thomas Monjalon wrote:
> 06/01/2021 10:06, David Marchand:
>> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
>> <maxime.coquelin@redhat.com> wrote:
>>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
>>> index 1f1f63a1a5..f4775ff141 100644
>>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>>> @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
>>> char *mac_addr = NULL;
>>> int ret = -1;
>>>
>>> + /*
>>> + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
>>> + * it implies API breakage.
>>
>> Extending rte_vdev_driver to implement this detection would be an ABI breakage.
>> This is a driver-only API (rte_vdev_driver is only used by the vdev
>> bus and drivers afaics).
>>
>> Doing this is allowed as per my understanding of the ABI policy which
>> guarantees ABI stability for applications.
>> We do not guarantee this stability for OOT drivers.
>
> I agree.
> As a reminder, the A in ABI stands for Application.
>
+1, as long as the binary interface remains the same, we are good.
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user
2021-01-06 9:06 ` David Marchand
2021-01-06 9:11 ` Thomas Monjalon
@ 2021-01-06 9:14 ` Maxime Coquelin
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-06 9:14 UTC (permalink / raw)
To: David Marchand, Ray Kinsella, Thomas Monjalon
Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 10:06 AM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
>> index 1f1f63a1a5..f4775ff141 100644
>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>> @@ -663,6 +663,17 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
>> char *mac_addr = NULL;
>> int ret = -1;
>>
>> + /*
>> + * ToDo 1: Implement detection mechanism at vdev bus level as PCI, but
>> + * it implies API breakage.
>
> Extending rte_vdev_driver to implement this detection would be an ABI breakage.
> This is a driver-only API (rte_vdev_driver is only used by the vdev
> bus and drivers afaics).
>
> Doing this is allowed as per my understanding of the ABI policy which
> guarantees ABI stability for applications.
> We do not guarantee this stability for OOT drivers.
>
That would be a good news, as it would remove impacting the user by
requiring him to manually add --iova-mode=va in the EAL parameters.
I can change this in the v2 if this is confirmed. Ray, Thomas, is that
OK for you?
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (7 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 08/40] net/virtio: force IOVA as VA mode for Virtio-user Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:07 ` Xia, Chenbo
2021-01-06 9:14 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing Maxime Coquelin
` (31 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Going further in making the Virtio ethdev layer bus agnostic,
this patch adds a boolean in the Virtio PCI device metadata.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_pci.c | 20 ++++++++++++--------
drivers/net/virtio/virtio_pci.h | 3 ++-
drivers/net/virtio/virtio_pci_ethdev.c | 12 +++++++-----
3 files changed, 21 insertions(+), 14 deletions(-)
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 8605254e53..7f0c066968 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -687,26 +687,29 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
* Return 0 on success.
*/
int
-vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
+vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
{
+ struct virtio_hw *hw = &dev->hw;
+
/*
* Try if we can succeed reading virtio pci caps, which exists
* only on modern pci device. If failed, we fallback to legacy
* virtio handling.
*/
- if (virtio_read_caps(dev, hw) == 0) {
+ if (virtio_read_caps(pci_dev, hw) == 0) {
PMD_INIT_LOG(INFO, "modern virtio pci detected.");
virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
hw->bus_type = VIRTIO_BUS_PCI_MODERN;
+ dev->modern = true;
goto msix_detect;
}
PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
- if (rte_pci_ioport_map(dev, 0, VTPCI_IO(hw)) < 0) {
- rte_pci_unmap_device(dev);
- if (dev->kdrv == RTE_PCI_KDRV_UNKNOWN &&
- (!dev->device.devargs ||
- dev->device.devargs->bus !=
+ if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0) {
+ rte_pci_unmap_device(pci_dev);
+ if (pci_dev->kdrv == RTE_PCI_KDRV_UNKNOWN &&
+ (!pci_dev->device.devargs ||
+ pci_dev->device.devargs->bus !=
rte_bus_find_by_name("pci"))) {
PMD_INIT_LOG(INFO,
"skip kernel managed virtio device.");
@@ -717,9 +720,10 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
+ dev->modern = false;
msix_detect:
- hw->use_msix = vtpci_msix_detect(dev);
+ hw->use_msix = vtpci_msix_detect(pci_dev);
return 0;
}
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 8d3dc0e22e..3e245ed630 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -291,6 +291,7 @@ struct virtio_hw {
struct virtio_pci_dev {
struct virtio_hw hw;
+ bool modern;
};
#define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
@@ -367,7 +368,7 @@ vtpci_packed_queue(struct virtio_hw *hw)
/*
* Function declaration from virtio_pci.c
*/
-int vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw);
+int vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev);
void vtpci_reset(struct virtio_hw *);
void vtpci_reinit_complete(struct virtio_hw *);
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
index d6cbe582d2..f513381707 100644
--- a/drivers/net/virtio/virtio_pci_ethdev.c
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -39,9 +39,11 @@ static const struct rte_pci_id pci_id_virtio_map[] = {
* could have the PCI initiated correctly.
*/
static int
-virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
+virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
{
- if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
+ struct virtio_hw *hw = &dev->hw;
+
+ if (dev->modern) {
/*
* We don't have to re-parse the PCI config space, since
* rte_pci_map_device() makes sure the mapped address
@@ -57,7 +59,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
PMD_INIT_LOG(DEBUG, "failed to map pci device!");
return -1;
}
- } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
+ } else {
if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
return -1;
}
@@ -74,13 +76,13 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
int ret;
if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
- ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
+ ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
if (ret) {
PMD_INIT_LOG(ERR, "Failed to init PCI device\n");
return -1;
}
} else {
- ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
+ ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
if (ret < 0) {
PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
return -1;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata
2020-12-20 21:13 ` [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata Maxime Coquelin
@ 2020-12-30 3:07 ` Xia, Chenbo
2021-01-06 9:14 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:07 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata
>
> Going further in making the Virtio ethdev layer bus agnostic,
> this patch adds a boolean in the Virtio PCI device metadata.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_pci.c | 20 ++++++++++++--------
> drivers/net/virtio/virtio_pci.h | 3 ++-
> drivers/net/virtio/virtio_pci_ethdev.c | 12 +++++++-----
> 3 files changed, 21 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 8605254e53..7f0c066968 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -687,26 +687,29 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
> * Return 0 on success.
> */
> int
> -vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw)
> +vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
> {
> + struct virtio_hw *hw = &dev->hw;
> +
> /*
> * Try if we can succeed reading virtio pci caps, which exists
> * only on modern pci device. If failed, we fallback to legacy
> * virtio handling.
> */
> - if (virtio_read_caps(dev, hw) == 0) {
> + if (virtio_read_caps(pci_dev, hw) == 0) {
> PMD_INIT_LOG(INFO, "modern virtio pci detected.");
> virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
> hw->bus_type = VIRTIO_BUS_PCI_MODERN;
> + dev->modern = true;
> goto msix_detect;
> }
>
> PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
> - if (rte_pci_ioport_map(dev, 0, VTPCI_IO(hw)) < 0) {
> - rte_pci_unmap_device(dev);
> - if (dev->kdrv == RTE_PCI_KDRV_UNKNOWN &&
> - (!dev->device.devargs ||
> - dev->device.devargs->bus !=
> + if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0) {
> + rte_pci_unmap_device(pci_dev);
> + if (pci_dev->kdrv == RTE_PCI_KDRV_UNKNOWN &&
> + (!pci_dev->device.devargs ||
> + pci_dev->device.devargs->bus !=
> rte_bus_find_by_name("pci"))) {
> PMD_INIT_LOG(INFO,
> "skip kernel managed virtio device.");
> @@ -717,9 +720,10 @@ vtpci_init(struct rte_pci_device *dev, struct virtio_hw
> *hw)
>
> virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
> hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
> + dev->modern = false;
>
> msix_detect:
> - hw->use_msix = vtpci_msix_detect(dev);
> + hw->use_msix = vtpci_msix_detect(pci_dev);
>
> return 0;
> }
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 8d3dc0e22e..3e245ed630 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -291,6 +291,7 @@ struct virtio_hw {
>
> struct virtio_pci_dev {
> struct virtio_hw hw;
> + bool modern;
> };
>
> #define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
> @@ -367,7 +368,7 @@ vtpci_packed_queue(struct virtio_hw *hw)
> /*
> * Function declaration from virtio_pci.c
> */
> -int vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw);
> +int vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev);
> void vtpci_reset(struct virtio_hw *);
>
> void vtpci_reinit_complete(struct virtio_hw *);
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> index d6cbe582d2..f513381707 100644
> --- a/drivers/net/virtio/virtio_pci_ethdev.c
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -39,9 +39,11 @@ static const struct rte_pci_id pci_id_virtio_map[] = {
> * could have the PCI initiated correctly.
> */
> static int
> -virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> +virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
> {
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN) {
> + struct virtio_hw *hw = &dev->hw;
> +
> + if (dev->modern) {
> /*
> * We don't have to re-parse the PCI config space, since
> * rte_pci_map_device() makes sure the mapped address
> @@ -57,7 +59,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct
> virtio_hw *hw)
> PMD_INIT_LOG(DEBUG, "failed to map pci device!");
> return -1;
> }
> - } else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY) {
> + } else {
> if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
> return -1;
> }
> @@ -74,13 +76,13 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> int ret;
>
> if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> - ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> + ret = vtpci_init(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
> if (ret) {
> PMD_INIT_LOG(ERR, "Failed to init PCI device\n");
> return -1;
> }
> } else {
> - ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), hw);
> + ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
> if (ret < 0) {
> PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
> return -1;
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata
2020-12-20 21:13 ` [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata Maxime Coquelin
2020-12-30 3:07 ` Xia, Chenbo
@ 2021-01-06 9:14 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:14 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> Going further in making the Virtio ethdev layer bus agnostic,
> this patch adds a boolean in the Virtio PCI device metadata.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (8 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 09/40] net/virtio: store PCI type in Virtio device metadata Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:07 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level Maxime Coquelin
` (30 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduces a new callback for device closing,
making virtio_dev_close() bus-agnostic.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/meson.build | 2 --
drivers/net/virtio/virtio_ethdev.c | 13 +------------
drivers/net/virtio/virtio_pci.c | 25 +++++++++++++++++++++++++
drivers/net/virtio/virtio_pci.h | 2 ++
drivers/net/virtio/virtio_user_ethdev.c | 11 +++++++++++
5 files changed, 39 insertions(+), 14 deletions(-)
diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
index 8e0f1a9951..0b62418f33 100644
--- a/drivers/net/virtio/meson.build
+++ b/drivers/net/virtio/meson.build
@@ -37,8 +37,6 @@ elif arch_subdir == 'arm' and host_machine.cpu_family().startswith('aarch64')
endif
if is_linux
- dpdk_conf.set('RTE_VIRTIO_USER', 1)
-
sources += files('virtio_user_ethdev.c',
'virtio_user/vhost_kernel.c',
'virtio_user/vhost_kernel_tap.c',
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 13e2ec998a..00aa38e4ef 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -718,18 +718,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
virtio_dev_free_mbufs(dev);
virtio_free_queues(hw);
-#ifdef RTE_VIRTIO_USER
- if (hw->bus_type == VIRTIO_BUS_USER)
- virtio_user_dev_uninit(dev->data->dev_private);
- else
-#endif
- if (dev->device) {
- rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(dev));
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
- rte_pci_ioport_unmap(VTPCI_IO(hw));
- }
-
- return 0;
+ return VTPCI_OPS(hw)->dev_close(hw);
}
static int
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 7f0c066968..599d8afa6b 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -241,6 +241,17 @@ legacy_notify_queue(struct virtio_hw *hw, struct virtqueue *vq)
VIRTIO_PCI_QUEUE_NOTIFY);
}
+static int
+legacy_dev_close(struct virtio_hw *hw)
+{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_pci_unmap_device(dev->pci_dev);
+ rte_pci_ioport_unmap(VTPCI_IO(hw));
+
+ return 0;
+}
+
const struct virtio_pci_ops legacy_ops = {
.read_dev_cfg = legacy_read_dev_config,
.write_dev_cfg = legacy_write_dev_config,
@@ -255,6 +266,7 @@ const struct virtio_pci_ops legacy_ops = {
.setup_queue = legacy_setup_queue,
.del_queue = legacy_del_queue,
.notify_queue = legacy_notify_queue,
+ .dev_close = legacy_dev_close,
};
static inline void
@@ -446,6 +458,16 @@ modern_notify_queue(struct virtio_hw *hw, struct virtqueue *vq)
rte_write32(notify_data, vq->notify_addr);
}
+static int
+modern_dev_close(struct virtio_hw *hw)
+{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_pci_unmap_device(dev->pci_dev);
+
+ return 0;
+}
+
const struct virtio_pci_ops modern_ops = {
.read_dev_cfg = modern_read_dev_config,
.write_dev_cfg = modern_write_dev_config,
@@ -460,6 +482,7 @@ const struct virtio_pci_ops modern_ops = {
.setup_queue = modern_setup_queue,
.del_queue = modern_del_queue,
.notify_queue = modern_notify_queue,
+ .dev_close = modern_dev_close,
};
@@ -691,6 +714,8 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
{
struct virtio_hw *hw = &dev->hw;
+ dev->pci_dev = pci_dev;
+
/*
* Try if we can succeed reading virtio pci caps, which exists
* only on modern pci device. If failed, we fallback to legacy
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 3e245ed630..4f7d0e479e 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -239,6 +239,7 @@ struct virtio_pci_ops {
int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
+ int (*dev_close)(struct virtio_hw *hw);
};
struct virtio_net_config;
@@ -291,6 +292,7 @@ struct virtio_hw {
struct virtio_pci_dev {
struct virtio_hw hw;
+ struct rte_pci_device *pci_dev;
bool modern;
};
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index f4775ff141..f9a2dbae71 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -462,6 +462,16 @@ virtio_user_notify_queue(struct virtio_hw *hw, struct virtqueue *vq)
strerror(errno));
}
+static int
+virtio_user_dev_close(struct virtio_hw *hw)
+{
+ struct virtio_user_dev *dev = virtio_user_get_dev(hw);
+
+ virtio_user_dev_uninit(dev);
+
+ return 0;
+}
+
const struct virtio_pci_ops virtio_user_ops = {
.read_dev_cfg = virtio_user_read_dev_config,
.write_dev_cfg = virtio_user_write_dev_config,
@@ -476,6 +486,7 @@ const struct virtio_pci_ops virtio_user_ops = {
.setup_queue = virtio_user_setup_queue,
.del_queue = virtio_user_del_queue,
.notify_queue = virtio_user_notify_queue,
+ .dev_close = virtio_user_dev_close,
};
static const char *valid_args[] = {
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing
2020-12-20 21:13 ` [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing Maxime Coquelin
@ 2020-12-30 3:07 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:07 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 10/40] net/virtio: add callback for device closing
>
> This patch introduces a new callback for device closing,
> making virtio_dev_close() bus-agnostic.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/meson.build | 2 --
> drivers/net/virtio/virtio_ethdev.c | 13 +------------
> drivers/net/virtio/virtio_pci.c | 25 +++++++++++++++++++++++++
> drivers/net/virtio/virtio_pci.h | 2 ++
> drivers/net/virtio/virtio_user_ethdev.c | 11 +++++++++++
> 5 files changed, 39 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
> index 8e0f1a9951..0b62418f33 100644
> --- a/drivers/net/virtio/meson.build
> +++ b/drivers/net/virtio/meson.build
> @@ -37,8 +37,6 @@ elif arch_subdir == 'arm' and
> host_machine.cpu_family().startswith('aarch64')
> endif
>
> if is_linux
> - dpdk_conf.set('RTE_VIRTIO_USER', 1)
> -
> sources += files('virtio_user_ethdev.c',
> 'virtio_user/vhost_kernel.c',
> 'virtio_user/vhost_kernel_tap.c',
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 13e2ec998a..00aa38e4ef 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -718,18 +718,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
> virtio_dev_free_mbufs(dev);
> virtio_free_queues(hw);
>
> -#ifdef RTE_VIRTIO_USER
> - if (hw->bus_type == VIRTIO_BUS_USER)
> - virtio_user_dev_uninit(dev->data->dev_private);
> - else
> -#endif
> - if (dev->device) {
> - rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(dev));
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> - rte_pci_ioport_unmap(VTPCI_IO(hw));
> - }
> -
> - return 0;
> + return VTPCI_OPS(hw)->dev_close(hw);
> }
>
> static int
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 7f0c066968..599d8afa6b 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -241,6 +241,17 @@ legacy_notify_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> VIRTIO_PCI_QUEUE_NOTIFY);
> }
>
> +static int
> +legacy_dev_close(struct virtio_hw *hw)
> +{
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_pci_unmap_device(dev->pci_dev);
> + rte_pci_ioport_unmap(VTPCI_IO(hw));
> +
> + return 0;
> +}
> +
> const struct virtio_pci_ops legacy_ops = {
> .read_dev_cfg = legacy_read_dev_config,
> .write_dev_cfg = legacy_write_dev_config,
> @@ -255,6 +266,7 @@ const struct virtio_pci_ops legacy_ops = {
> .setup_queue = legacy_setup_queue,
> .del_queue = legacy_del_queue,
> .notify_queue = legacy_notify_queue,
> + .dev_close = legacy_dev_close,
> };
>
> static inline void
> @@ -446,6 +458,16 @@ modern_notify_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> rte_write32(notify_data, vq->notify_addr);
> }
>
> +static int
> +modern_dev_close(struct virtio_hw *hw)
> +{
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_pci_unmap_device(dev->pci_dev);
> +
> + return 0;
> +}
> +
> const struct virtio_pci_ops modern_ops = {
> .read_dev_cfg = modern_read_dev_config,
> .write_dev_cfg = modern_write_dev_config,
> @@ -460,6 +482,7 @@ const struct virtio_pci_ops modern_ops = {
> .setup_queue = modern_setup_queue,
> .del_queue = modern_del_queue,
> .notify_queue = modern_notify_queue,
> + .dev_close = modern_dev_close,
> };
>
>
> @@ -691,6 +714,8 @@ vtpci_init(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> {
> struct virtio_hw *hw = &dev->hw;
>
> + dev->pci_dev = pci_dev;
> +
> /*
> * Try if we can succeed reading virtio pci caps, which exists
> * only on modern pci device. If failed, we fallback to legacy
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 3e245ed630..4f7d0e479e 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -239,6 +239,7 @@ struct virtio_pci_ops {
> int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + int (*dev_close)(struct virtio_hw *hw);
> };
>
> struct virtio_net_config;
> @@ -291,6 +292,7 @@ struct virtio_hw {
>
> struct virtio_pci_dev {
> struct virtio_hw hw;
> + struct rte_pci_device *pci_dev;
> bool modern;
> };
>
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index f4775ff141..f9a2dbae71 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -462,6 +462,16 @@ virtio_user_notify_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> strerror(errno));
> }
>
> +static int
> +virtio_user_dev_close(struct virtio_hw *hw)
> +{
> + struct virtio_user_dev *dev = virtio_user_get_dev(hw);
> +
> + virtio_user_dev_uninit(dev);
> +
> + return 0;
> +}
> +
> const struct virtio_pci_ops virtio_user_ops = {
> .read_dev_cfg = virtio_user_read_dev_config,
> .write_dev_cfg = virtio_user_write_dev_config,
> @@ -476,6 +486,7 @@ const struct virtio_pci_ops virtio_user_ops = {
> .setup_queue = virtio_user_setup_queue,
> .del_queue = virtio_user_del_queue,
> .notify_queue = virtio_user_notify_queue,
> + .dev_close = virtio_user_dev_close,
> };
>
> static const char *valid_args[] = {
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing
2020-12-20 21:13 ` [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing Maxime Coquelin
2020-12-30 3:07 ` Xia, Chenbo
@ 2021-01-06 9:33 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:33 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:14 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch introduces a new callback for device closing,
> making virtio_dev_close() bus-agnostic.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (9 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 10/40] net/virtio: add callback for device closing Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum Maxime Coquelin
` (29 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch provides a new callback for the bus type
to validate negotiated features are compatible with it.
Only user for now is PCI modern bus type, which implies
that the device supports Virtio 1.0+.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 11 +++++------
drivers/net/virtio/virtio_pci.c | 19 +++++++++++++++++++
drivers/net/virtio/virtio_pci.h | 1 +
drivers/net/virtio/virtio_user_ethdev.c | 7 +++++++
4 files changed, 32 insertions(+), 6 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 00aa38e4ef..91a93b2b6e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1315,17 +1315,16 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
hw->guest_features);
- if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
- PMD_INIT_LOG(ERR,
- "VIRTIO_F_VERSION_1 features is not enabled.");
+ if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
+ PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
return -1;
}
- if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_USER) {
+ if (vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
+
if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
- PMD_INIT_LOG(ERR,
- "failed to set FEATURES_OK status!");
+ PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
return -1;
}
}
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 599d8afa6b..3de7980b4f 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -151,6 +151,12 @@ legacy_set_features(struct virtio_hw *hw, uint64_t features)
VIRTIO_PCI_GUEST_FEATURES);
}
+static int
+legacy_features_ok(struct virtio_hw *hw __rte_unused)
+{
+ return 0;
+}
+
static uint8_t
legacy_get_status(struct virtio_hw *hw)
{
@@ -259,6 +265,7 @@ const struct virtio_pci_ops legacy_ops = {
.set_status = legacy_set_status,
.get_features = legacy_get_features,
.set_features = legacy_set_features,
+ .features_ok = legacy_features_ok,
.get_isr = legacy_get_isr,
.set_config_irq = legacy_set_config_irq,
.set_queue_irq = legacy_set_queue_irq,
@@ -332,6 +339,17 @@ modern_set_features(struct virtio_hw *hw, uint64_t features)
&hw->common_cfg->guest_feature);
}
+static int
+modern_features_ok(struct virtio_hw *hw)
+{
+ if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
+ PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
+ return -1;
+ }
+
+ return 0;
+}
+
static uint8_t
modern_get_status(struct virtio_hw *hw)
{
@@ -475,6 +493,7 @@ const struct virtio_pci_ops modern_ops = {
.set_status = modern_set_status,
.get_features = modern_get_features,
.set_features = modern_set_features,
+ .features_ok = modern_features_ok,
.get_isr = modern_get_isr,
.set_config_irq = modern_set_config_irq,
.set_queue_irq = modern_set_queue_irq,
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 4f7d0e479e..22c21e6896 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -227,6 +227,7 @@ struct virtio_pci_ops {
uint64_t (*get_features)(struct virtio_hw *hw);
void (*set_features)(struct virtio_hw *hw, uint64_t features);
+ int (*features_ok)(struct virtio_hw *hw);
uint8_t (*get_isr)(struct virtio_hw *hw);
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index f9a2dbae71..c93e0e43f5 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -327,6 +327,12 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
dev->features = features & dev->device_features;
}
+static int
+virtio_user_features_ok(struct virtio_hw *hw __rte_unused)
+{
+ return 0;
+}
+
static uint8_t
virtio_user_get_isr(struct virtio_hw *hw __rte_unused)
{
@@ -479,6 +485,7 @@ const struct virtio_pci_ops virtio_user_ops = {
.set_status = virtio_user_set_status,
.get_features = virtio_user_get_features,
.set_features = virtio_user_set_features,
+ .features_ok = virtio_user_features_ok,
.get_isr = virtio_user_get_isr,
.set_config_irq = virtio_user_set_config_irq,
.set_queue_irq = virtio_user_set_queue_irq,
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level
2020-12-20 21:13 ` [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level Maxime Coquelin
@ 2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:08 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 11/40] net/virtio: validate features at bus level
>
> This patch provides a new callback for the bus type
> to validate negotiated features are compatible with it.
>
> Only user for now is PCI modern bus type, which implies
> that the device supports Virtio 1.0+.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 11 +++++------
> drivers/net/virtio/virtio_pci.c | 19 +++++++++++++++++++
> drivers/net/virtio/virtio_pci.h | 1 +
> drivers/net/virtio/virtio_user_ethdev.c | 7 +++++++
> 4 files changed, 32 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 00aa38e4ef..91a93b2b6e 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1315,17 +1315,16 @@ virtio_negotiate_features(struct virtio_hw *hw,
> uint64_t req_features)
> PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
> hw->guest_features);
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw,
> VIRTIO_F_VERSION_1)) {
> - PMD_INIT_LOG(ERR,
> - "VIRTIO_F_VERSION_1 features is not enabled.");
> + if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
> + PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
> return -1;
> }
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type ==
> VIRTIO_BUS_USER) {
> + if (vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
> +
> if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
> - PMD_INIT_LOG(ERR,
> - "failed to set FEATURES_OK status!");
> + PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
> return -1;
> }
> }
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 599d8afa6b..3de7980b4f 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -151,6 +151,12 @@ legacy_set_features(struct virtio_hw *hw, uint64_t
> features)
> VIRTIO_PCI_GUEST_FEATURES);
> }
>
> +static int
> +legacy_features_ok(struct virtio_hw *hw __rte_unused)
> +{
> + return 0;
> +}
> +
> static uint8_t
> legacy_get_status(struct virtio_hw *hw)
> {
> @@ -259,6 +265,7 @@ const struct virtio_pci_ops legacy_ops = {
> .set_status = legacy_set_status,
> .get_features = legacy_get_features,
> .set_features = legacy_set_features,
> + .features_ok = legacy_features_ok,
> .get_isr = legacy_get_isr,
> .set_config_irq = legacy_set_config_irq,
> .set_queue_irq = legacy_set_queue_irq,
> @@ -332,6 +339,17 @@ modern_set_features(struct virtio_hw *hw, uint64_t
> features)
> &hw->common_cfg->guest_feature);
> }
>
> +static int
> +modern_features_ok(struct virtio_hw *hw)
> +{
> + if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> + PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> static uint8_t
> modern_get_status(struct virtio_hw *hw)
> {
> @@ -475,6 +493,7 @@ const struct virtio_pci_ops modern_ops = {
> .set_status = modern_set_status,
> .get_features = modern_get_features,
> .set_features = modern_set_features,
> + .features_ok = modern_features_ok,
> .get_isr = modern_get_isr,
> .set_config_irq = modern_set_config_irq,
> .set_queue_irq = modern_set_queue_irq,
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 4f7d0e479e..22c21e6896 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -227,6 +227,7 @@ struct virtio_pci_ops {
>
> uint64_t (*get_features)(struct virtio_hw *hw);
> void (*set_features)(struct virtio_hw *hw, uint64_t features);
> + int (*features_ok)(struct virtio_hw *hw);
>
> uint8_t (*get_isr)(struct virtio_hw *hw);
>
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index f9a2dbae71..c93e0e43f5 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -327,6 +327,12 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t
> features)
> dev->features = features & dev->device_features;
> }
>
> +static int
> +virtio_user_features_ok(struct virtio_hw *hw __rte_unused)
> +{
> + return 0;
> +}
> +
> static uint8_t
> virtio_user_get_isr(struct virtio_hw *hw __rte_unused)
> {
> @@ -479,6 +485,7 @@ const struct virtio_pci_ops virtio_user_ops = {
> .set_status = virtio_user_set_status,
> .get_features = virtio_user_get_features,
> .set_features = virtio_user_set_features,
> + .features_ok = virtio_user_features_ok,
> .get_isr = virtio_user_get_isr,
> .set_config_irq = virtio_user_set_config_irq,
> .set_queue_irq = virtio_user_set_queue_irq,
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level
2020-12-20 21:13 ` [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
@ 2021-01-06 9:33 ` David Marchand
2021-01-15 9:20 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:33 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index 00aa38e4ef..91a93b2b6e 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1315,17 +1315,16 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
> PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
> hw->guest_features);
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> - PMD_INIT_LOG(ERR,
> - "VIRTIO_F_VERSION_1 features is not enabled.");
> + if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
> + PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
We have a log which gives more context in the modern features_ok() callback.
I don't think we need both log messages.
> return -1;
> }
>
> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_USER) {
> + if (vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
> +
> if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
> - PMD_INIT_LOG(ERR,
> - "failed to set FEATURES_OK status!");
> + PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
> return -1;
> }
> }
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 599d8afa6b..3de7980b4f 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
[snip]
> @@ -332,6 +339,17 @@ modern_set_features(struct virtio_hw *hw, uint64_t features)
> &hw->common_cfg->guest_feature);
> }
>
> +static int
> +modern_features_ok(struct virtio_hw *hw)
> +{
> + if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> + PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> static uint8_t
> modern_get_status(struct virtio_hw *hw)
> {
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level
2021-01-06 9:33 ` David Marchand
@ 2021-01-15 9:20 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 9:20 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 10:33 AM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
>> index 00aa38e4ef..91a93b2b6e 100644
>> --- a/drivers/net/virtio/virtio_ethdev.c
>> +++ b/drivers/net/virtio/virtio_ethdev.c
>> @@ -1315,17 +1315,16 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
>> PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
>> hw->guest_features);
>>
>> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN && !vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
>> - PMD_INIT_LOG(ERR,
>> - "VIRTIO_F_VERSION_1 features is not enabled.");
>> + if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
>> + PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
>
> We have a log which gives more context in the modern features_ok() callback.
> I don't think we need both log messages.
Yes, maybe overkill. I will remove.
Thanks,
Maxime
>> return -1;
>> }
>>
>> - if (hw->bus_type == VIRTIO_BUS_PCI_MODERN || hw->bus_type == VIRTIO_BUS_USER) {
>> + if (vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
>> vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
>> +
>> if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
>> - PMD_INIT_LOG(ERR,
>> - "failed to set FEATURES_OK status!");
>> + PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
>> return -1;
>> }
>> }
>> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
>> index 599d8afa6b..3de7980b4f 100644
>> --- a/drivers/net/virtio/virtio_pci.c
>> +++ b/drivers/net/virtio/virtio_pci.c
>
> [snip]
>
>
>> @@ -332,6 +339,17 @@ modern_set_features(struct virtio_hw *hw, uint64_t features)
>> &hw->common_cfg->guest_feature);
>> }
>>
>> +static int
>> +modern_features_ok(struct virtio_hw *hw)
>> +{
>> + if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
>> + PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
>> + return -1;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static uint8_t
>> modern_get_status(struct virtio_hw *hw)
>> {
>
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (10 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 11/40] net/virtio: validate features at bus level Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device Maxime Coquelin
` (28 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Bus type awareness at the generic ethdev level is no
more needed as previous patches have made it bus-agnostic.
This patch removes it from struct virtio_hw.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 15 ---------------
drivers/net/virtio/virtio_pci.c | 2 --
drivers/net/virtio/virtio_pci.h | 8 --------
drivers/net/virtio/virtio_pci_ethdev.c | 7 ++++++-
drivers/net/virtio/virtio_user_ethdev.c | 5 ++++-
5 files changed, 10 insertions(+), 27 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 91a93b2b6e..86d8930e78 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1785,20 +1785,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
return 0;
}
-
-static void
-virtio_set_vtpci_ops(struct virtio_hw *hw)
-{
- if (hw->bus_type == VIRTIO_BUS_USER)
- VTPCI_OPS(hw) = &virtio_user_ops;
- else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
- VTPCI_OPS(hw) = &modern_ops;
- else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
- VTPCI_OPS(hw) = &legacy_ops;
-
- return;
-}
-
/*
* This function is based on probe() function in virtio_pci.c
* It returns 0 on success.
@@ -1824,7 +1810,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
- virtio_set_vtpci_ops(hw);
set_rxtx_funcs(eth_dev);
return 0;
}
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 3de7980b4f..f9331e37d7 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -743,7 +743,6 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
if (virtio_read_caps(pci_dev, hw) == 0) {
PMD_INIT_LOG(INFO, "modern virtio pci detected.");
virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
- hw->bus_type = VIRTIO_BUS_PCI_MODERN;
dev->modern = true;
goto msix_detect;
}
@@ -763,7 +762,6 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
}
virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
- hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
dev->modern = false;
msix_detect:
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 22c21e6896..a60edc4a93 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -245,15 +245,7 @@ struct virtio_pci_ops {
struct virtio_net_config;
-enum virtio_bus_type {
- VIRTIO_BUS_UNKNOWN,
- VIRTIO_BUS_PCI_LEGACY,
- VIRTIO_BUS_PCI_MODERN,
- VIRTIO_BUS_USER,
-};
-
struct virtio_hw {
- enum virtio_bus_type bus_type;
struct virtnet_ctl *cvq;
uint64_t req_guest_features;
uint64_t guest_features;
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
index f513381707..a6d5e2e158 100644
--- a/drivers/net/virtio/virtio_pci_ethdev.c
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -82,6 +82,11 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
return -1;
}
} else {
+ if (dev->modern)
+ VTPCI_OPS(hw) = &modern_ops;
+ else
+ VTPCI_OPS(hw) = &legacy_ops;
+
ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
if (ret < 0) {
PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
@@ -103,7 +108,7 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
err_unmap:
rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
- if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
+ if (!dev->modern)
rte_pci_ioport_unmap(VTPCI_IO(hw));
return ret;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index c93e0e43f5..1420db32be 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -641,7 +641,6 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
* Here just pretend that we support msix.
*/
hw->use_msix = 1;
- hw->bus_type = VIRTIO_BUS_USER;
hw->use_vec_rx = 0;
hw->use_vec_tx = 0;
hw->use_inorder_rx = 0;
@@ -700,6 +699,10 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
return -1;
}
+ dev = eth_dev->data->dev_private;
+ hw = &dev->hw;
+ VTPCI_OPS(hw) = &virtio_user_ops;
+
if (eth_virtio_dev_init(eth_dev) < 0) {
PMD_INIT_LOG(ERR, "eth_virtio_dev_init fails");
rte_eth_dev_release_port(eth_dev);
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum
2020-12-20 21:13 ` [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum Maxime Coquelin
@ 2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:33 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:08 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 12/40] net/virtio: remove bus type enum
>
> Bus type awareness at the generic ethdev level is no
> more needed as previous patches have made it bus-agnostic.
>
> This patch removes it from struct virtio_hw.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 15 ---------------
> drivers/net/virtio/virtio_pci.c | 2 --
> drivers/net/virtio/virtio_pci.h | 8 --------
> drivers/net/virtio/virtio_pci_ethdev.c | 7 ++++++-
> drivers/net/virtio/virtio_user_ethdev.c | 5 ++++-
> 5 files changed, 10 insertions(+), 27 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 91a93b2b6e..86d8930e78 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1785,20 +1785,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> return 0;
> }
>
> -
> -static void
> -virtio_set_vtpci_ops(struct virtio_hw *hw)
> -{
> - if (hw->bus_type == VIRTIO_BUS_USER)
> - VTPCI_OPS(hw) = &virtio_user_ops;
> - else if (hw->bus_type == VIRTIO_BUS_PCI_MODERN)
> - VTPCI_OPS(hw) = &modern_ops;
> - else if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> - VTPCI_OPS(hw) = &legacy_ops;
> -
> - return;
> -}
> -
> /*
> * This function is based on probe() function in virtio_pci.c
> * It returns 0 on success.
> @@ -1824,7 +1810,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> eth_dev->rx_descriptor_done = virtio_dev_rx_queue_done;
>
> if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> - virtio_set_vtpci_ops(hw);
> set_rxtx_funcs(eth_dev);
> return 0;
> }
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 3de7980b4f..f9331e37d7 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -743,7 +743,6 @@ vtpci_init(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> if (virtio_read_caps(pci_dev, hw) == 0) {
> PMD_INIT_LOG(INFO, "modern virtio pci detected.");
> virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
> - hw->bus_type = VIRTIO_BUS_PCI_MODERN;
> dev->modern = true;
> goto msix_detect;
> }
> @@ -763,7 +762,6 @@ vtpci_init(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> }
>
> virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
> - hw->bus_type = VIRTIO_BUS_PCI_LEGACY;
> dev->modern = false;
>
> msix_detect:
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 22c21e6896..a60edc4a93 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -245,15 +245,7 @@ struct virtio_pci_ops {
>
> struct virtio_net_config;
>
> -enum virtio_bus_type {
> - VIRTIO_BUS_UNKNOWN,
> - VIRTIO_BUS_PCI_LEGACY,
> - VIRTIO_BUS_PCI_MODERN,
> - VIRTIO_BUS_USER,
> -};
> -
> struct virtio_hw {
> - enum virtio_bus_type bus_type;
> struct virtnet_ctl *cvq;
> uint64_t req_guest_features;
> uint64_t guest_features;
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> index f513381707..a6d5e2e158 100644
> --- a/drivers/net/virtio/virtio_pci_ethdev.c
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -82,6 +82,11 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> return -1;
> }
> } else {
> + if (dev->modern)
> + VTPCI_OPS(hw) = &modern_ops;
> + else
> + VTPCI_OPS(hw) = &legacy_ops;
> +
> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
> if (ret < 0) {
> PMD_INIT_LOG(ERR, "Failed to remap PCI device\n");
> @@ -103,7 +108,7 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
>
> err_unmap:
> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> - if (hw->bus_type == VIRTIO_BUS_PCI_LEGACY)
> + if (!dev->modern)
> rte_pci_ioport_unmap(VTPCI_IO(hw));
>
> return ret;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index c93e0e43f5..1420db32be 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -641,7 +641,6 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
> * Here just pretend that we support msix.
> */
> hw->use_msix = 1;
> - hw->bus_type = VIRTIO_BUS_USER;
> hw->use_vec_rx = 0;
> hw->use_vec_tx = 0;
> hw->use_inorder_rx = 0;
> @@ -700,6 +699,10 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
> return -1;
> }
>
> + dev = eth_dev->data->dev_private;
> + hw = &dev->hw;
> + VTPCI_OPS(hw) = &virtio_user_ops;
> +
> if (eth_virtio_dev_init(eth_dev) < 0) {
> PMD_INIT_LOG(ERR, "eth_virtio_dev_init fails");
> rte_eth_dev_release_port(eth_dev);
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum
2020-12-20 21:13 ` [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
@ 2021-01-06 9:33 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:33 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> Bus type awareness at the generic ethdev level is no
> more needed as previous patches have made it bus-agnostic.
>
> This patch removes it from struct virtio_hw.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (11 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 12/40] net/virtio: remove bus type enum Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:58 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct Maxime Coquelin
` (27 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves the fields from virtio_hw structure that
are PCI-specific to virtio_pci_dev_struct.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_pci.c | 139 ++++++++++++++++++--------------
drivers/net/virtio/virtio_pci.h | 10 +--
2 files changed, 85 insertions(+), 64 deletions(-)
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index f9331e37d7..8c62507a0a 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -287,18 +287,19 @@ static void
modern_read_dev_config(struct virtio_hw *hw, size_t offset,
void *dst, int length)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
int i;
uint8_t *p;
uint8_t old_gen, new_gen;
do {
- old_gen = rte_read8(&hw->common_cfg->config_generation);
+ old_gen = rte_read8(&dev->common_cfg->config_generation);
p = dst;
for (i = 0; i < length; i++)
- *p++ = rte_read8((uint8_t *)hw->dev_cfg + offset + i);
+ *p++ = rte_read8((uint8_t *)dev->dev_cfg + offset + i);
- new_gen = rte_read8(&hw->common_cfg->config_generation);
+ new_gen = rte_read8(&dev->common_cfg->config_generation);
} while (old_gen != new_gen);
}
@@ -306,23 +307,25 @@ static void
modern_write_dev_config(struct virtio_hw *hw, size_t offset,
const void *src, int length)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
int i;
const uint8_t *p = src;
for (i = 0; i < length; i++)
- rte_write8((*p++), (((uint8_t *)hw->dev_cfg) + offset + i));
+ rte_write8((*p++), (((uint8_t *)dev->dev_cfg) + offset + i));
}
static uint64_t
modern_get_features(struct virtio_hw *hw)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
uint32_t features_lo, features_hi;
- rte_write32(0, &hw->common_cfg->device_feature_select);
- features_lo = rte_read32(&hw->common_cfg->device_feature);
+ rte_write32(0, &dev->common_cfg->device_feature_select);
+ features_lo = rte_read32(&dev->common_cfg->device_feature);
- rte_write32(1, &hw->common_cfg->device_feature_select);
- features_hi = rte_read32(&hw->common_cfg->device_feature);
+ rte_write32(1, &dev->common_cfg->device_feature_select);
+ features_hi = rte_read32(&dev->common_cfg->device_feature);
return ((uint64_t)features_hi << 32) | features_lo;
}
@@ -330,13 +333,15 @@ modern_get_features(struct virtio_hw *hw)
static void
modern_set_features(struct virtio_hw *hw, uint64_t features)
{
- rte_write32(0, &hw->common_cfg->guest_feature_select);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write32(0, &dev->common_cfg->guest_feature_select);
rte_write32(features & ((1ULL << 32) - 1),
- &hw->common_cfg->guest_feature);
+ &dev->common_cfg->guest_feature);
- rte_write32(1, &hw->common_cfg->guest_feature_select);
+ rte_write32(1, &dev->common_cfg->guest_feature_select);
rte_write32(features >> 32,
- &hw->common_cfg->guest_feature);
+ &dev->common_cfg->guest_feature);
}
static int
@@ -353,46 +358,59 @@ modern_features_ok(struct virtio_hw *hw)
static uint8_t
modern_get_status(struct virtio_hw *hw)
{
- return rte_read8(&hw->common_cfg->device_status);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ return rte_read8(&dev->common_cfg->device_status);
}
static void
modern_set_status(struct virtio_hw *hw, uint8_t status)
{
- rte_write8(status, &hw->common_cfg->device_status);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write8(status, &dev->common_cfg->device_status);
}
static uint8_t
modern_get_isr(struct virtio_hw *hw)
{
- return rte_read8(hw->isr);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ return rte_read8(dev->isr);
}
static uint16_t
modern_set_config_irq(struct virtio_hw *hw, uint16_t vec)
{
- rte_write16(vec, &hw->common_cfg->msix_config);
- return rte_read16(&hw->common_cfg->msix_config);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write16(vec, &dev->common_cfg->msix_config);
+ return rte_read16(&dev->common_cfg->msix_config);
}
static uint16_t
modern_set_queue_irq(struct virtio_hw *hw, struct virtqueue *vq, uint16_t vec)
{
- rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
- rte_write16(vec, &hw->common_cfg->queue_msix_vector);
- return rte_read16(&hw->common_cfg->queue_msix_vector);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
+ rte_write16(vec, &dev->common_cfg->queue_msix_vector);
+ return rte_read16(&dev->common_cfg->queue_msix_vector);
}
static uint16_t
modern_get_queue_num(struct virtio_hw *hw, uint16_t queue_id)
{
- rte_write16(queue_id, &hw->common_cfg->queue_select);
- return rte_read16(&hw->common_cfg->queue_size);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write16(queue_id, &dev->common_cfg->queue_select);
+ return rte_read16(&dev->common_cfg->queue_size);
}
static int
modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
uint64_t desc_addr, avail_addr, used_addr;
uint16_t notify_off;
@@ -405,20 +423,20 @@ modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
ring[vq->vq_nentries]),
VIRTIO_PCI_VRING_ALIGN);
- rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
+ rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
- io_write64_twopart(desc_addr, &hw->common_cfg->queue_desc_lo,
- &hw->common_cfg->queue_desc_hi);
- io_write64_twopart(avail_addr, &hw->common_cfg->queue_avail_lo,
- &hw->common_cfg->queue_avail_hi);
- io_write64_twopart(used_addr, &hw->common_cfg->queue_used_lo,
- &hw->common_cfg->queue_used_hi);
+ io_write64_twopart(desc_addr, &dev->common_cfg->queue_desc_lo,
+ &dev->common_cfg->queue_desc_hi);
+ io_write64_twopart(avail_addr, &dev->common_cfg->queue_avail_lo,
+ &dev->common_cfg->queue_avail_hi);
+ io_write64_twopart(used_addr, &dev->common_cfg->queue_used_lo,
+ &dev->common_cfg->queue_used_hi);
- notify_off = rte_read16(&hw->common_cfg->queue_notify_off);
- vq->notify_addr = (void *)((uint8_t *)hw->notify_base +
- notify_off * hw->notify_off_multiplier);
+ notify_off = rte_read16(&dev->common_cfg->queue_notify_off);
+ vq->notify_addr = (void *)((uint8_t *)dev->notify_base +
+ notify_off * dev->notify_off_multiplier);
- rte_write16(1, &hw->common_cfg->queue_enable);
+ rte_write16(1, &dev->common_cfg->queue_enable);
PMD_INIT_LOG(DEBUG, "queue %u addresses:", vq->vq_queue_index);
PMD_INIT_LOG(DEBUG, "\t desc_addr: %" PRIx64, desc_addr);
@@ -433,16 +451,18 @@ modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
static void
modern_del_queue(struct virtio_hw *hw, struct virtqueue *vq)
{
- rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
- io_write64_twopart(0, &hw->common_cfg->queue_desc_lo,
- &hw->common_cfg->queue_desc_hi);
- io_write64_twopart(0, &hw->common_cfg->queue_avail_lo,
- &hw->common_cfg->queue_avail_hi);
- io_write64_twopart(0, &hw->common_cfg->queue_used_lo,
- &hw->common_cfg->queue_used_hi);
+ io_write64_twopart(0, &dev->common_cfg->queue_desc_lo,
+ &dev->common_cfg->queue_desc_hi);
+ io_write64_twopart(0, &dev->common_cfg->queue_avail_lo,
+ &dev->common_cfg->queue_avail_hi);
+ io_write64_twopart(0, &dev->common_cfg->queue_used_lo,
+ &dev->common_cfg->queue_used_hi);
- rte_write16(0, &hw->common_cfg->queue_enable);
+ rte_write16(0, &dev->common_cfg->queue_enable);
}
static void
@@ -607,18 +627,19 @@ get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap)
#define PCI_MSIX_ENABLE 0x8000
static int
-virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
+virtio_read_caps(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
uint8_t pos;
struct virtio_pci_cap cap;
int ret;
- if (rte_pci_map_device(dev)) {
+ if (rte_pci_map_device(pci_dev)) {
PMD_INIT_LOG(DEBUG, "failed to map pci device!");
return -1;
}
- ret = rte_pci_read_config(dev, &pos, 1, PCI_CAPABILITY_LIST);
+ ret = rte_pci_read_config(pci_dev, &pos, 1, PCI_CAPABILITY_LIST);
if (ret != 1) {
PMD_INIT_LOG(DEBUG,
"failed to read pci capability list, ret %d", ret);
@@ -626,7 +647,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
}
while (pos) {
- ret = rte_pci_read_config(dev, &cap, 2, pos);
+ ret = rte_pci_read_config(pci_dev, &cap, 2, pos);
if (ret != 2) {
PMD_INIT_LOG(DEBUG,
"failed to read pci cap at pos: %x ret %d",
@@ -642,7 +663,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
*/
uint16_t flags;
- ret = rte_pci_read_config(dev, &flags, sizeof(flags),
+ ret = rte_pci_read_config(pci_dev, &flags, sizeof(flags),
pos + 2);
if (ret != sizeof(flags)) {
PMD_INIT_LOG(DEBUG,
@@ -664,7 +685,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
goto next;
}
- ret = rte_pci_read_config(dev, &cap, sizeof(cap), pos);
+ ret = rte_pci_read_config(pci_dev, &cap, sizeof(cap), pos);
if (ret != sizeof(cap)) {
PMD_INIT_LOG(DEBUG,
"failed to read pci cap at pos: %x ret %d",
@@ -678,24 +699,24 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
switch (cap.cfg_type) {
case VIRTIO_PCI_CAP_COMMON_CFG:
- hw->common_cfg = get_cfg_addr(dev, &cap);
+ dev->common_cfg = get_cfg_addr(pci_dev, &cap);
break;
case VIRTIO_PCI_CAP_NOTIFY_CFG:
- ret = rte_pci_read_config(dev,
- &hw->notify_off_multiplier,
+ ret = rte_pci_read_config(pci_dev,
+ &dev->notify_off_multiplier,
4, pos + sizeof(cap));
if (ret != 4)
PMD_INIT_LOG(DEBUG,
"failed to read notify_off_multiplier, ret %d",
ret);
else
- hw->notify_base = get_cfg_addr(dev, &cap);
+ dev->notify_base = get_cfg_addr(pci_dev, &cap);
break;
case VIRTIO_PCI_CAP_DEVICE_CFG:
- hw->dev_cfg = get_cfg_addr(dev, &cap);
+ dev->dev_cfg = get_cfg_addr(pci_dev, &cap);
break;
case VIRTIO_PCI_CAP_ISR_CFG:
- hw->isr = get_cfg_addr(dev, &cap);
+ dev->isr = get_cfg_addr(pci_dev, &cap);
break;
}
@@ -703,19 +724,19 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
pos = cap.cap_next;
}
- if (hw->common_cfg == NULL || hw->notify_base == NULL ||
- hw->dev_cfg == NULL || hw->isr == NULL) {
+ if (dev->common_cfg == NULL || dev->notify_base == NULL ||
+ dev->dev_cfg == NULL || dev->isr == NULL) {
PMD_INIT_LOG(INFO, "no modern virtio pci device found.");
return -1;
}
PMD_INIT_LOG(INFO, "found modern virtio pci device.");
- PMD_INIT_LOG(DEBUG, "common cfg mapped at: %p", hw->common_cfg);
- PMD_INIT_LOG(DEBUG, "device cfg mapped at: %p", hw->dev_cfg);
- PMD_INIT_LOG(DEBUG, "isr cfg mapped at: %p", hw->isr);
+ PMD_INIT_LOG(DEBUG, "common cfg mapped at: %p", dev->common_cfg);
+ PMD_INIT_LOG(DEBUG, "device cfg mapped at: %p", dev->dev_cfg);
+ PMD_INIT_LOG(DEBUG, "isr cfg mapped at: %p", dev->isr);
PMD_INIT_LOG(DEBUG, "notify base: %p, notify off multiplier: %u",
- hw->notify_base, hw->notify_off_multiplier);
+ dev->notify_base, dev->notify_off_multiplier);
return 0;
}
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index a60edc4a93..5629b37050 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -264,13 +264,8 @@ struct virtio_hw {
bool has_rx_offload;
uint16_t port_id;
uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
- uint32_t notify_off_multiplier;
uint32_t speed; /* link speed in MB */
uint8_t duplex;
- uint8_t *isr;
- uint16_t *notify_base;
- struct virtio_pci_common_cfg *common_cfg;
- struct virtio_net_config *dev_cfg;
/*
* App management thread and virtio interrupt handler thread
* both can change device state, this lock is meant to avoid
@@ -286,6 +281,11 @@ struct virtio_hw {
struct virtio_pci_dev {
struct virtio_hw hw;
struct rte_pci_device *pci_dev;
+ struct virtio_pci_common_cfg *common_cfg;
+ struct virtio_net_config *dev_cfg;
+ uint8_t *isr;
+ uint16_t *notify_base;
+ uint32_t notify_off_multiplier;
bool modern;
};
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device
2020-12-20 21:13 ` [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device Maxime Coquelin
@ 2020-12-30 3:08 ` Xia, Chenbo
2021-01-06 9:58 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:08 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device
>
> This patch moves the fields from virtio_hw structure that
> are PCI-specific to virtio_pci_dev_struct.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_pci.c | 139 ++++++++++++++++++--------------
> drivers/net/virtio/virtio_pci.h | 10 +--
> 2 files changed, 85 insertions(+), 64 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index f9331e37d7..8c62507a0a 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -287,18 +287,19 @@ static void
> modern_read_dev_config(struct virtio_hw *hw, size_t offset,
> void *dst, int length)
> {
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> int i;
> uint8_t *p;
> uint8_t old_gen, new_gen;
>
> do {
> - old_gen = rte_read8(&hw->common_cfg->config_generation);
> + old_gen = rte_read8(&dev->common_cfg->config_generation);
>
> p = dst;
> for (i = 0; i < length; i++)
> - *p++ = rte_read8((uint8_t *)hw->dev_cfg + offset + i);
> + *p++ = rte_read8((uint8_t *)dev->dev_cfg + offset + i);
>
> - new_gen = rte_read8(&hw->common_cfg->config_generation);
> + new_gen = rte_read8(&dev->common_cfg->config_generation);
> } while (old_gen != new_gen);
> }
>
> @@ -306,23 +307,25 @@ static void
> modern_write_dev_config(struct virtio_hw *hw, size_t offset,
> const void *src, int length)
> {
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> int i;
> const uint8_t *p = src;
>
> for (i = 0; i < length; i++)
> - rte_write8((*p++), (((uint8_t *)hw->dev_cfg) + offset + i));
> + rte_write8((*p++), (((uint8_t *)dev->dev_cfg) + offset + i));
> }
>
> static uint64_t
> modern_get_features(struct virtio_hw *hw)
> {
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> uint32_t features_lo, features_hi;
>
> - rte_write32(0, &hw->common_cfg->device_feature_select);
> - features_lo = rte_read32(&hw->common_cfg->device_feature);
> + rte_write32(0, &dev->common_cfg->device_feature_select);
> + features_lo = rte_read32(&dev->common_cfg->device_feature);
>
> - rte_write32(1, &hw->common_cfg->device_feature_select);
> - features_hi = rte_read32(&hw->common_cfg->device_feature);
> + rte_write32(1, &dev->common_cfg->device_feature_select);
> + features_hi = rte_read32(&dev->common_cfg->device_feature);
>
> return ((uint64_t)features_hi << 32) | features_lo;
> }
> @@ -330,13 +333,15 @@ modern_get_features(struct virtio_hw *hw)
> static void
> modern_set_features(struct virtio_hw *hw, uint64_t features)
> {
> - rte_write32(0, &hw->common_cfg->guest_feature_select);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write32(0, &dev->common_cfg->guest_feature_select);
> rte_write32(features & ((1ULL << 32) - 1),
> - &hw->common_cfg->guest_feature);
> + &dev->common_cfg->guest_feature);
>
> - rte_write32(1, &hw->common_cfg->guest_feature_select);
> + rte_write32(1, &dev->common_cfg->guest_feature_select);
> rte_write32(features >> 32,
> - &hw->common_cfg->guest_feature);
> + &dev->common_cfg->guest_feature);
> }
>
> static int
> @@ -353,46 +358,59 @@ modern_features_ok(struct virtio_hw *hw)
> static uint8_t
> modern_get_status(struct virtio_hw *hw)
> {
> - return rte_read8(&hw->common_cfg->device_status);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + return rte_read8(&dev->common_cfg->device_status);
> }
>
> static void
> modern_set_status(struct virtio_hw *hw, uint8_t status)
> {
> - rte_write8(status, &hw->common_cfg->device_status);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write8(status, &dev->common_cfg->device_status);
> }
>
> static uint8_t
> modern_get_isr(struct virtio_hw *hw)
> {
> - return rte_read8(hw->isr);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + return rte_read8(dev->isr);
> }
>
> static uint16_t
> modern_set_config_irq(struct virtio_hw *hw, uint16_t vec)
> {
> - rte_write16(vec, &hw->common_cfg->msix_config);
> - return rte_read16(&hw->common_cfg->msix_config);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write16(vec, &dev->common_cfg->msix_config);
> + return rte_read16(&dev->common_cfg->msix_config);
> }
>
> static uint16_t
> modern_set_queue_irq(struct virtio_hw *hw, struct virtqueue *vq, uint16_t vec)
> {
> - rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
> - rte_write16(vec, &hw->common_cfg->queue_msix_vector);
> - return rte_read16(&hw->common_cfg->queue_msix_vector);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
> + rte_write16(vec, &dev->common_cfg->queue_msix_vector);
> + return rte_read16(&dev->common_cfg->queue_msix_vector);
> }
>
> static uint16_t
> modern_get_queue_num(struct virtio_hw *hw, uint16_t queue_id)
> {
> - rte_write16(queue_id, &hw->common_cfg->queue_select);
> - return rte_read16(&hw->common_cfg->queue_size);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write16(queue_id, &dev->common_cfg->queue_select);
> + return rte_read16(&dev->common_cfg->queue_size);
> }
>
> static int
> modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
> {
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> uint64_t desc_addr, avail_addr, used_addr;
> uint16_t notify_off;
>
> @@ -405,20 +423,20 @@ modern_setup_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> ring[vq->vq_nentries]),
> VIRTIO_PCI_VRING_ALIGN);
>
> - rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
> + rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
>
> - io_write64_twopart(desc_addr, &hw->common_cfg->queue_desc_lo,
> - &hw->common_cfg->queue_desc_hi);
> - io_write64_twopart(avail_addr, &hw->common_cfg->queue_avail_lo,
> - &hw->common_cfg->queue_avail_hi);
> - io_write64_twopart(used_addr, &hw->common_cfg->queue_used_lo,
> - &hw->common_cfg->queue_used_hi);
> + io_write64_twopart(desc_addr, &dev->common_cfg->queue_desc_lo,
> + &dev->common_cfg->queue_desc_hi);
> + io_write64_twopart(avail_addr, &dev->common_cfg->queue_avail_lo,
> + &dev->common_cfg->queue_avail_hi);
> + io_write64_twopart(used_addr, &dev->common_cfg->queue_used_lo,
> + &dev->common_cfg->queue_used_hi);
>
> - notify_off = rte_read16(&hw->common_cfg->queue_notify_off);
> - vq->notify_addr = (void *)((uint8_t *)hw->notify_base +
> - notify_off * hw->notify_off_multiplier);
> + notify_off = rte_read16(&dev->common_cfg->queue_notify_off);
> + vq->notify_addr = (void *)((uint8_t *)dev->notify_base +
> + notify_off * dev->notify_off_multiplier);
>
> - rte_write16(1, &hw->common_cfg->queue_enable);
> + rte_write16(1, &dev->common_cfg->queue_enable);
>
> PMD_INIT_LOG(DEBUG, "queue %u addresses:", vq->vq_queue_index);
> PMD_INIT_LOG(DEBUG, "\t desc_addr: %" PRIx64, desc_addr);
> @@ -433,16 +451,18 @@ modern_setup_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> static void
> modern_del_queue(struct virtio_hw *hw, struct virtqueue *vq)
> {
> - rte_write16(vq->vq_queue_index, &hw->common_cfg->queue_select);
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
>
> - io_write64_twopart(0, &hw->common_cfg->queue_desc_lo,
> - &hw->common_cfg->queue_desc_hi);
> - io_write64_twopart(0, &hw->common_cfg->queue_avail_lo,
> - &hw->common_cfg->queue_avail_hi);
> - io_write64_twopart(0, &hw->common_cfg->queue_used_lo,
> - &hw->common_cfg->queue_used_hi);
> + io_write64_twopart(0, &dev->common_cfg->queue_desc_lo,
> + &dev->common_cfg->queue_desc_hi);
> + io_write64_twopart(0, &dev->common_cfg->queue_avail_lo,
> + &dev->common_cfg->queue_avail_hi);
> + io_write64_twopart(0, &dev->common_cfg->queue_used_lo,
> + &dev->common_cfg->queue_used_hi);
>
> - rte_write16(0, &hw->common_cfg->queue_enable);
> + rte_write16(0, &dev->common_cfg->queue_enable);
> }
>
> static void
> @@ -607,18 +627,19 @@ get_cfg_addr(struct rte_pci_device *dev, struct
> virtio_pci_cap *cap)
> #define PCI_MSIX_ENABLE 0x8000
>
> static int
> -virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
> +virtio_read_caps(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
> {
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> uint8_t pos;
> struct virtio_pci_cap cap;
> int ret;
>
> - if (rte_pci_map_device(dev)) {
> + if (rte_pci_map_device(pci_dev)) {
> PMD_INIT_LOG(DEBUG, "failed to map pci device!");
> return -1;
> }
>
> - ret = rte_pci_read_config(dev, &pos, 1, PCI_CAPABILITY_LIST);
> + ret = rte_pci_read_config(pci_dev, &pos, 1, PCI_CAPABILITY_LIST);
> if (ret != 1) {
> PMD_INIT_LOG(DEBUG,
> "failed to read pci capability list, ret %d", ret);
> @@ -626,7 +647,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
> }
>
> while (pos) {
> - ret = rte_pci_read_config(dev, &cap, 2, pos);
> + ret = rte_pci_read_config(pci_dev, &cap, 2, pos);
> if (ret != 2) {
> PMD_INIT_LOG(DEBUG,
> "failed to read pci cap at pos: %x ret %d",
> @@ -642,7 +663,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
> */
> uint16_t flags;
>
> - ret = rte_pci_read_config(dev, &flags, sizeof(flags),
> + ret = rte_pci_read_config(pci_dev, &flags, sizeof(flags),
> pos + 2);
> if (ret != sizeof(flags)) {
> PMD_INIT_LOG(DEBUG,
> @@ -664,7 +685,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
> goto next;
> }
>
> - ret = rte_pci_read_config(dev, &cap, sizeof(cap), pos);
> + ret = rte_pci_read_config(pci_dev, &cap, sizeof(cap), pos);
> if (ret != sizeof(cap)) {
> PMD_INIT_LOG(DEBUG,
> "failed to read pci cap at pos: %x ret %d",
> @@ -678,24 +699,24 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
>
> switch (cap.cfg_type) {
> case VIRTIO_PCI_CAP_COMMON_CFG:
> - hw->common_cfg = get_cfg_addr(dev, &cap);
> + dev->common_cfg = get_cfg_addr(pci_dev, &cap);
> break;
> case VIRTIO_PCI_CAP_NOTIFY_CFG:
> - ret = rte_pci_read_config(dev,
> - &hw->notify_off_multiplier,
> + ret = rte_pci_read_config(pci_dev,
> + &dev->notify_off_multiplier,
> 4, pos + sizeof(cap));
> if (ret != 4)
> PMD_INIT_LOG(DEBUG,
> "failed to read notify_off_multiplier, ret %d",
> ret);
> else
> - hw->notify_base = get_cfg_addr(dev, &cap);
> + dev->notify_base = get_cfg_addr(pci_dev, &cap);
> break;
> case VIRTIO_PCI_CAP_DEVICE_CFG:
> - hw->dev_cfg = get_cfg_addr(dev, &cap);
> + dev->dev_cfg = get_cfg_addr(pci_dev, &cap);
> break;
> case VIRTIO_PCI_CAP_ISR_CFG:
> - hw->isr = get_cfg_addr(dev, &cap);
> + dev->isr = get_cfg_addr(pci_dev, &cap);
> break;
> }
>
> @@ -703,19 +724,19 @@ virtio_read_caps(struct rte_pci_device *dev, struct
> virtio_hw *hw)
> pos = cap.cap_next;
> }
>
> - if (hw->common_cfg == NULL || hw->notify_base == NULL ||
> - hw->dev_cfg == NULL || hw->isr == NULL) {
> + if (dev->common_cfg == NULL || dev->notify_base == NULL ||
> + dev->dev_cfg == NULL || dev->isr == NULL) {
> PMD_INIT_LOG(INFO, "no modern virtio pci device found.");
> return -1;
> }
>
> PMD_INIT_LOG(INFO, "found modern virtio pci device.");
>
> - PMD_INIT_LOG(DEBUG, "common cfg mapped at: %p", hw->common_cfg);
> - PMD_INIT_LOG(DEBUG, "device cfg mapped at: %p", hw->dev_cfg);
> - PMD_INIT_LOG(DEBUG, "isr cfg mapped at: %p", hw->isr);
> + PMD_INIT_LOG(DEBUG, "common cfg mapped at: %p", dev->common_cfg);
> + PMD_INIT_LOG(DEBUG, "device cfg mapped at: %p", dev->dev_cfg);
> + PMD_INIT_LOG(DEBUG, "isr cfg mapped at: %p", dev->isr);
> PMD_INIT_LOG(DEBUG, "notify base: %p, notify off multiplier: %u",
> - hw->notify_base, hw->notify_off_multiplier);
> + dev->notify_base, dev->notify_off_multiplier);
>
> return 0;
> }
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index a60edc4a93..5629b37050 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -264,13 +264,8 @@ struct virtio_hw {
> bool has_rx_offload;
> uint16_t port_id;
> uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> - uint32_t notify_off_multiplier;
> uint32_t speed; /* link speed in MB */
> uint8_t duplex;
> - uint8_t *isr;
> - uint16_t *notify_base;
> - struct virtio_pci_common_cfg *common_cfg;
> - struct virtio_net_config *dev_cfg;
> /*
> * App management thread and virtio interrupt handler thread
> * both can change device state, this lock is meant to avoid
> @@ -286,6 +281,11 @@ struct virtio_hw {
> struct virtio_pci_dev {
> struct virtio_hw hw;
> struct rte_pci_device *pci_dev;
> + struct virtio_pci_common_cfg *common_cfg;
> + struct virtio_net_config *dev_cfg;
> + uint8_t *isr;
> + uint16_t *notify_base;
> + uint32_t notify_off_multiplier;
> bool modern;
> };
>
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device
2020-12-20 21:13 ` [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device Maxime Coquelin
2020-12-30 3:08 ` Xia, Chenbo
@ 2021-01-06 9:58 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:58 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves the fields from virtio_hw structure that
> are PCI-specific to virtio_pci_dev_struct.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (12 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 13/40] net/virtio: move PCI-specific fields to PCI device Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 9:58 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI Maxime Coquelin
` (26 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch improves the virtio_hw struct packing,
going from 88 down to 80 bytes with a 6 bytes hole in
the end of the first cacheline. Fields only used in the
slow path are placed in the end, so that hot path only
uses the first cacheline.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_pci.h | 45 ++++++++++++++++-----------------
1 file changed, 22 insertions(+), 23 deletions(-)
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 5629b37050..15f68f141c 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -246,26 +246,25 @@ struct virtio_pci_ops {
struct virtio_net_config;
struct virtio_hw {
- struct virtnet_ctl *cvq;
- uint64_t req_guest_features;
- uint64_t guest_features;
- uint32_t max_queue_pairs;
- bool started;
- uint16_t max_mtu;
- uint16_t vtnet_hdr_size;
- uint8_t vlan_strip;
- uint8_t use_msix;
- uint8_t use_vec_rx;
- uint8_t use_vec_tx;
- uint8_t use_inorder_rx;
- uint8_t use_inorder_tx;
- uint8_t weak_barriers;
- bool has_tx_offload;
- bool has_rx_offload;
- uint16_t port_id;
- uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
- uint32_t speed; /* link speed in MB */
- uint8_t duplex;
+ struct virtqueue **vqs;
+ uint64_t guest_features;
+ uint16_t vtnet_hdr_size;
+ uint8_t started;
+ uint8_t weak_barriers;
+ uint8_t vlan_strip;
+ uint8_t has_tx_offload;
+ uint8_t has_rx_offload;
+ uint8_t use_vec_rx;
+ uint8_t use_vec_tx;
+ uint8_t use_inorder_rx;
+ uint8_t use_inorder_tx;
+ uint8_t opened;
+ uint16_t port_id;
+ uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
+ uint32_t speed; /* link speed in MB */
+ uint8_t duplex;
+ uint8_t use_msix;
+ uint16_t max_mtu;
/*
* App management thread and virtio interrupt handler thread
* both can change device state, this lock is meant to avoid
@@ -273,9 +272,9 @@ struct virtio_hw {
*/
rte_spinlock_t state_lock;
struct rte_mbuf **inject_pkts;
- bool opened;
-
- struct virtqueue **vqs;
+ uint16_t max_queue_pairs;
+ uint64_t req_guest_features;
+ struct virtnet_ctl *cvq;
};
struct virtio_pci_dev {
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct
2020-12-20 21:13 ` [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct Maxime Coquelin
@ 2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 9:58 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:09 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 14/40] net/virtio: pack virtio HW struct
>
> This patch improves the virtio_hw struct packing,
> going from 88 down to 80 bytes with a 6 bytes hole in
> the end of the first cacheline. Fields only used in the
> slow path are placed in the end, so that hot path only
> uses the first cacheline.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_pci.h | 45 ++++++++++++++++-----------------
> 1 file changed, 22 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 5629b37050..15f68f141c 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -246,26 +246,25 @@ struct virtio_pci_ops {
> struct virtio_net_config;
>
> struct virtio_hw {
> - struct virtnet_ctl *cvq;
> - uint64_t req_guest_features;
> - uint64_t guest_features;
> - uint32_t max_queue_pairs;
> - bool started;
> - uint16_t max_mtu;
> - uint16_t vtnet_hdr_size;
> - uint8_t vlan_strip;
> - uint8_t use_msix;
> - uint8_t use_vec_rx;
> - uint8_t use_vec_tx;
> - uint8_t use_inorder_rx;
> - uint8_t use_inorder_tx;
> - uint8_t weak_barriers;
> - bool has_tx_offload;
> - bool has_rx_offload;
> - uint16_t port_id;
> - uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> - uint32_t speed; /* link speed in MB */
> - uint8_t duplex;
> + struct virtqueue **vqs;
> + uint64_t guest_features;
> + uint16_t vtnet_hdr_size;
> + uint8_t started;
> + uint8_t weak_barriers;
> + uint8_t vlan_strip;
> + uint8_t has_tx_offload;
> + uint8_t has_rx_offload;
> + uint8_t use_vec_rx;
> + uint8_t use_vec_tx;
> + uint8_t use_inorder_rx;
> + uint8_t use_inorder_tx;
> + uint8_t opened;
> + uint16_t port_id;
> + uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> + uint32_t speed; /* link speed in MB */
> + uint8_t duplex;
> + uint8_t use_msix;
> + uint16_t max_mtu;
> /*
> * App management thread and virtio interrupt handler thread
> * both can change device state, this lock is meant to avoid
> @@ -273,9 +272,9 @@ struct virtio_hw {
> */
> rte_spinlock_t state_lock;
> struct rte_mbuf **inject_pkts;
> - bool opened;
> -
> - struct virtqueue **vqs;
> + uint16_t max_queue_pairs;
> + uint64_t req_guest_features;
> + struct virtnet_ctl *cvq;
> };
>
> struct virtio_pci_dev {
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct
2020-12-20 21:13 ` [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
@ 2021-01-06 9:58 ` David Marchand
2021-01-15 9:35 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 9:58 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch improves the virtio_hw struct packing,
> going from 88 down to 80 bytes with a 6 bytes hole in
> the end of the first cacheline. Fields only used in the
> slow path are placed in the end, so that hot path only
> uses the first cacheline.
This also changes some boolean fields to uint8_t but we still assign
the true/false values to them (in some cases).
This works, but I would align those fields' assignments to this type change.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct
2021-01-06 9:58 ` David Marchand
@ 2021-01-15 9:35 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 9:35 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 10:58 AM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch improves the virtio_hw struct packing,
>> going from 88 down to 80 bytes with a 6 bytes hole in
>> the end of the first cacheline. Fields only used in the
>> slow path are placed in the end, so that hot path only
>> uses the first cacheline.
>
> This also changes some boolean fields to uint8_t but we still assign
> the true/false values to them (in some cases).
> This works, but I would align those fields' assignments to this type change.
>
Fixed, note it was already inconsistent for some.
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (13 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 14/40] net/virtio: pack virtio HW struct Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 10:09 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header Maxime Coquelin
` (25 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves Virtio PCI legacy IO handling to
virtio_pci.c. Two functions are created so that
virtio_pci_ethdev does not have to care about it.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_pci.c | 21 +++++++++++++++++++++
drivers/net/virtio/virtio_pci.h | 6 +++---
drivers/net/virtio/virtio_pci_ethdev.c | 4 ++--
3 files changed, 26 insertions(+), 5 deletions(-)
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 8c62507a0a..230a438bf7 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -31,6 +31,15 @@
#define VIRTIO_PCI_CONFIG(hw) \
(((hw)->use_msix == VIRTIO_MSIX_ENABLED) ? 24 : 20)
+
+struct virtio_pci_internal {
+ struct rte_pci_ioport io;
+};
+
+#define VTPCI_IO(hw) (&virtio_pci_internal[(hw)->port_id].io)
+
+struct virtio_pci_internal virtio_pci_internal[RTE_MAX_ETHPORTS];
+
static inline int
check_vq_phys_addr_ok(struct virtqueue *vq)
{
@@ -838,3 +847,15 @@ vtpci_msix_detect(struct rte_pci_device *dev)
return VIRTIO_MSIX_NONE;
}
+
+void vtpci_legacy_ioport_unmap(struct virtio_hw *hw)
+{
+ rte_pci_ioport_unmap(VTPCI_IO(hw));
+}
+
+int vtpci_legacy_ioport_map(struct virtio_hw *hw)
+{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
+
+ return rte_pci_ioport_map(dev->pci_dev, 0, VTPCI_IO(hw));
+}
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 15f68f141c..c3db36d2fc 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -297,15 +297,12 @@ struct virtio_pci_dev {
*/
struct virtio_hw_internal {
const struct virtio_pci_ops *vtpci_ops;
- struct rte_pci_ioport io;
};
#define VTPCI_OPS(hw) (virtio_hw_internal[(hw)->port_id].vtpci_ops)
-#define VTPCI_IO(hw) (&virtio_hw_internal[(hw)->port_id].io)
extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
-
/*
* This structure is just a reference to read
* net device specific config space; it just a chodu structure
@@ -380,6 +377,9 @@ uint8_t vtpci_isr(struct virtio_hw *);
enum virtio_msix_status vtpci_msix_detect(struct rte_pci_device *dev);
+void vtpci_legacy_ioport_unmap(struct virtio_hw *hw);
+int vtpci_legacy_ioport_map(struct virtio_hw *hw);
+
extern const struct virtio_pci_ops legacy_ops;
extern const struct virtio_pci_ops modern_ops;
extern const struct virtio_pci_ops virtio_user_ops;
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
index a6d5e2e158..17342ae7d8 100644
--- a/drivers/net/virtio/virtio_pci_ethdev.c
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -60,7 +60,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
return -1;
}
} else {
- if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
+ if (vtpci_legacy_ioport_map(hw) < 0)
return -1;
}
@@ -109,7 +109,7 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
err_unmap:
rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
if (!dev->modern)
- rte_pci_ioport_unmap(VTPCI_IO(hw));
+ vtpci_legacy_ioport_unmap(hw);
return ret;
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI
2020-12-20 21:13 ` [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI Maxime Coquelin
@ 2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 10:09 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:09 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI
>
> This patch moves Virtio PCI legacy IO handling to
> virtio_pci.c. Two functions are created so that
> virtio_pci_ethdev does not have to care about it.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_pci.c | 21 +++++++++++++++++++++
> drivers/net/virtio/virtio_pci.h | 6 +++---
> drivers/net/virtio/virtio_pci_ethdev.c | 4 ++--
> 3 files changed, 26 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 8c62507a0a..230a438bf7 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -31,6 +31,15 @@
> #define VIRTIO_PCI_CONFIG(hw) \
> (((hw)->use_msix == VIRTIO_MSIX_ENABLED) ? 24 : 20)
>
> +
> +struct virtio_pci_internal {
> + struct rte_pci_ioport io;
> +};
> +
> +#define VTPCI_IO(hw) (&virtio_pci_internal[(hw)->port_id].io)
> +
> +struct virtio_pci_internal virtio_pci_internal[RTE_MAX_ETHPORTS];
> +
> static inline int
> check_vq_phys_addr_ok(struct virtqueue *vq)
> {
> @@ -838,3 +847,15 @@ vtpci_msix_detect(struct rte_pci_device *dev)
>
> return VIRTIO_MSIX_NONE;
> }
> +
> +void vtpci_legacy_ioport_unmap(struct virtio_hw *hw)
> +{
> + rte_pci_ioport_unmap(VTPCI_IO(hw));
> +}
> +
> +int vtpci_legacy_ioport_map(struct virtio_hw *hw)
> +{
> + struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
> +
> + return rte_pci_ioport_map(dev->pci_dev, 0, VTPCI_IO(hw));
> +}
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 15f68f141c..c3db36d2fc 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -297,15 +297,12 @@ struct virtio_pci_dev {
> */
> struct virtio_hw_internal {
> const struct virtio_pci_ops *vtpci_ops;
> - struct rte_pci_ioport io;
> };
>
> #define VTPCI_OPS(hw) (virtio_hw_internal[(hw)->port_id].vtpci_ops)
> -#define VTPCI_IO(hw) (&virtio_hw_internal[(hw)->port_id].io)
>
> extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
>
> -
> /*
> * This structure is just a reference to read
> * net device specific config space; it just a chodu structure
> @@ -380,6 +377,9 @@ uint8_t vtpci_isr(struct virtio_hw *);
>
> enum virtio_msix_status vtpci_msix_detect(struct rte_pci_device *dev);
>
> +void vtpci_legacy_ioport_unmap(struct virtio_hw *hw);
> +int vtpci_legacy_ioport_map(struct virtio_hw *hw);
> +
> extern const struct virtio_pci_ops legacy_ops;
> extern const struct virtio_pci_ops modern_ops;
> extern const struct virtio_pci_ops virtio_user_ops;
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> index a6d5e2e158..17342ae7d8 100644
> --- a/drivers/net/virtio/virtio_pci_ethdev.c
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -60,7 +60,7 @@ virtio_remap_pci(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> return -1;
> }
> } else {
> - if (rte_pci_ioport_map(pci_dev, 0, VTPCI_IO(hw)) < 0)
> + if (vtpci_legacy_ioport_map(hw) < 0)
> return -1;
> }
>
> @@ -109,7 +109,7 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> err_unmap:
> rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev));
> if (!dev->modern)
> - rte_pci_ioport_unmap(VTPCI_IO(hw));
> + vtpci_legacy_ioport_unmap(hw);
>
> return ret;
> }
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI
2020-12-20 21:13 ` [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
@ 2021-01-06 10:09 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 10:09 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves Virtio PCI legacy IO handling to
> virtio_pci.c. Two functions are created so that
> virtio_pci_ethdev does not have to care about it.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (14 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 15/40] net/virtio: move legacy IO to Virtio PCI Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 10:08 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header Maxime Coquelin
` (24 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves virtio_hw and virtio callbacks into
a generic virtio header, now that they have been
curated from PCI references.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio.h | 75 ++++++++++++++++++++++
drivers/net/virtio/virtio_ethdev.c | 20 +++---
drivers/net/virtio/virtio_pci.c | 26 ++++----
drivers/net/virtio/virtio_pci.h | 82 ++-----------------------
drivers/net/virtio/virtio_pci_ethdev.c | 5 +-
drivers/net/virtio/virtio_user_ethdev.c | 8 +--
drivers/net/virtio/virtqueue.h | 2 +-
7 files changed, 110 insertions(+), 108 deletions(-)
create mode 100644 drivers/net/virtio/virtio.h
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
new file mode 100644
index 0000000000..eb078bc227
--- /dev/null
+++ b/drivers/net/virtio/virtio.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2020 Red Hat, Inc.
+ */
+
+#ifndef _VIRTIO_H_
+#define _VIRTIO_H_
+
+#include <rte_ether.h>
+
+struct virtio_hw {
+ struct virtqueue **vqs;
+ uint64_t guest_features;
+ uint16_t vtnet_hdr_size;
+ uint8_t started;
+ uint8_t weak_barriers;
+ uint8_t vlan_strip;
+ uint8_t has_tx_offload;
+ uint8_t has_rx_offload;
+ uint8_t use_vec_rx;
+ uint8_t use_vec_tx;
+ uint8_t use_inorder_rx;
+ uint8_t use_inorder_tx;
+ uint8_t opened;
+ uint16_t port_id;
+ uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
+ uint32_t speed; /* link speed in MB */
+ uint8_t duplex;
+ uint8_t use_msix;
+ uint16_t max_mtu;
+ /*
+ * App management thread and virtio interrupt handler thread
+ * both can change device state, this lock is meant to avoid
+ * such a contention.
+ */
+ rte_spinlock_t state_lock;
+ struct rte_mbuf **inject_pkts;
+ uint16_t max_queue_pairs;
+ uint64_t req_guest_features;
+ struct virtnet_ctl *cvq;
+};
+
+struct virtio_ops {
+ void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset, void *dst, int len);
+ void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset, const void *src, int len);
+ uint8_t (*get_status)(struct virtio_hw *hw);
+ void (*set_status)(struct virtio_hw *hw, uint8_t status);
+ uint64_t (*get_features)(struct virtio_hw *hw);
+ void (*set_features)(struct virtio_hw *hw, uint64_t features);
+ int (*features_ok)(struct virtio_hw *hw);
+ uint8_t (*get_isr)(struct virtio_hw *hw);
+ uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
+ uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq, uint16_t vec);
+ uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
+ int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
+ void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
+ void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
+ int (*dev_close)(struct virtio_hw *hw);
+};
+
+/*
+ * While virtio_hw is stored in shared memory, this structure stores
+ * some infos that may vary in the multiple process model locally.
+ * For example, the vtpci_ops pointer.
+ */
+struct virtio_hw_internal {
+ const struct virtio_ops *virtio_ops;
+};
+
+#define VIRTIO_OPS(hw) (virtio_hw_internal[(hw)->port_id].virtio_ops)
+
+extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
+
+
+#endif /* _VIRTIO_H_ */
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 86d8930e78..80b3fdba9e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -446,7 +446,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
* Read the virtqueue size from the Queue Size field
* Always power of 2 and if 0 virtqueue does not exist
*/
- vq_size = VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
+ vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
PMD_INIT_LOG(DEBUG, "vq_size: %u", vq_size);
if (vq_size == 0) {
PMD_INIT_LOG(ERR, "virtqueue does not exist");
@@ -608,7 +608,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
}
}
- if (VTPCI_OPS(hw)->setup_queue(hw, vq) < 0) {
+ if (VIRTIO_OPS(hw)->setup_queue(hw, vq) < 0) {
PMD_INIT_LOG(ERR, "setup_queue failed");
return -EINVAL;
}
@@ -703,7 +703,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
/* reset the NIC */
if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
- VTPCI_OPS(hw)->set_config_irq(hw, VIRTIO_MSI_NO_VECTOR);
+ VIRTIO_OPS(hw)->set_config_irq(hw, VIRTIO_MSI_NO_VECTOR);
if (intr_conf->rxq)
virtio_queues_unbind_intr(dev);
@@ -718,7 +718,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
virtio_dev_free_mbufs(dev);
virtio_free_queues(hw);
- return VTPCI_OPS(hw)->dev_close(hw);
+ return VIRTIO_OPS(hw)->dev_close(hw);
}
static int
@@ -1290,7 +1290,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
req_features);
/* Read device(host) feature bits */
- host_features = VTPCI_OPS(hw)->get_features(hw);
+ host_features = VIRTIO_OPS(hw)->get_features(hw);
PMD_INIT_LOG(DEBUG, "host_features before negotiate = %" PRIx64,
host_features);
@@ -1315,7 +1315,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
hw->guest_features);
- if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
+ if (VIRTIO_OPS(hw)->features_ok(hw) < 0) {
PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
return -1;
}
@@ -1551,7 +1551,7 @@ virtio_queues_bind_intr(struct rte_eth_dev *dev)
PMD_INIT_LOG(INFO, "queue/interrupt binding");
for (i = 0; i < dev->data->nb_rx_queues; ++i) {
dev->intr_handle->intr_vec[i] = i + 1;
- if (VTPCI_OPS(hw)->set_queue_irq(hw, hw->vqs[i * 2], i + 1) ==
+ if (VIRTIO_OPS(hw)->set_queue_irq(hw, hw->vqs[i * 2], i + 1) ==
VIRTIO_MSI_NO_VECTOR) {
PMD_DRV_LOG(ERR, "failed to set queue vector");
return -EBUSY;
@@ -1569,7 +1569,7 @@ virtio_queues_unbind_intr(struct rte_eth_dev *dev)
PMD_INIT_LOG(INFO, "queue/interrupt unbinding");
for (i = 0; i < dev->data->nb_rx_queues; ++i)
- VTPCI_OPS(hw)->set_queue_irq(hw,
+ VIRTIO_OPS(hw)->set_queue_irq(hw,
hw->vqs[i * VTNET_CQ],
VIRTIO_MSI_NO_VECTOR);
}
@@ -2081,7 +2081,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
/* Enable vector (0) for Link State Intrerrupt */
- if (VTPCI_OPS(hw)->set_config_irq(hw, 0) ==
+ if (VIRTIO_OPS(hw)->set_config_irq(hw, 0) ==
VIRTIO_MSI_NO_VECTOR) {
PMD_DRV_LOG(ERR, "failed to set config vector");
return -EBUSY;
@@ -2411,7 +2411,7 @@ virtio_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_rx_pktlen = VIRTIO_MAX_RX_PKTLEN;
dev_info->max_mac_addrs = VIRTIO_MAX_MAC_ADDRS;
- host_features = VTPCI_OPS(hw)->get_features(hw);
+ host_features = VIRTIO_OPS(hw)->get_features(hw);
dev_info->rx_offload_capa = DEV_RX_OFFLOAD_VLAN_STRIP;
dev_info->rx_offload_capa |= DEV_RX_OFFLOAD_JUMBO_FRAME;
if (host_features & (1ULL << VIRTIO_NET_F_GUEST_CSUM)) {
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 230a438bf7..df69fcdd45 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -267,7 +267,7 @@ legacy_dev_close(struct virtio_hw *hw)
return 0;
}
-const struct virtio_pci_ops legacy_ops = {
+const struct virtio_ops legacy_ops = {
.read_dev_cfg = legacy_read_dev_config,
.write_dev_cfg = legacy_write_dev_config,
.get_status = legacy_get_status,
@@ -515,7 +515,7 @@ modern_dev_close(struct virtio_hw *hw)
return 0;
}
-const struct virtio_pci_ops modern_ops = {
+const struct virtio_ops modern_ops = {
.read_dev_cfg = modern_read_dev_config,
.write_dev_cfg = modern_write_dev_config,
.get_status = modern_get_status,
@@ -538,14 +538,14 @@ void
vtpci_read_dev_config(struct virtio_hw *hw, size_t offset,
void *dst, int length)
{
- VTPCI_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
+ VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
}
void
vtpci_write_dev_config(struct virtio_hw *hw, size_t offset,
const void *src, int length)
{
- VTPCI_OPS(hw)->write_dev_cfg(hw, offset, src, length);
+ VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
}
uint64_t
@@ -558,7 +558,7 @@ vtpci_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
* host all support.
*/
features = host_features & hw->guest_features;
- VTPCI_OPS(hw)->set_features(hw, features);
+ VIRTIO_OPS(hw)->set_features(hw, features);
return features;
}
@@ -566,9 +566,9 @@ vtpci_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
void
vtpci_reset(struct virtio_hw *hw)
{
- VTPCI_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
+ VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
/* flush status write */
- VTPCI_OPS(hw)->get_status(hw);
+ VIRTIO_OPS(hw)->get_status(hw);
}
void
@@ -581,21 +581,21 @@ void
vtpci_set_status(struct virtio_hw *hw, uint8_t status)
{
if (status != VIRTIO_CONFIG_STATUS_RESET)
- status |= VTPCI_OPS(hw)->get_status(hw);
+ status |= VIRTIO_OPS(hw)->get_status(hw);
- VTPCI_OPS(hw)->set_status(hw, status);
+ VIRTIO_OPS(hw)->set_status(hw, status);
}
uint8_t
vtpci_get_status(struct virtio_hw *hw)
{
- return VTPCI_OPS(hw)->get_status(hw);
+ return VIRTIO_OPS(hw)->get_status(hw);
}
uint8_t
vtpci_isr(struct virtio_hw *hw)
{
- return VTPCI_OPS(hw)->get_isr(hw);
+ return VIRTIO_OPS(hw)->get_isr(hw);
}
static void *
@@ -772,7 +772,7 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
*/
if (virtio_read_caps(pci_dev, hw) == 0) {
PMD_INIT_LOG(INFO, "modern virtio pci detected.");
- virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
+ VIRTIO_OPS(hw) = &modern_ops;
dev->modern = true;
goto msix_detect;
}
@@ -791,7 +791,7 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
return -1;
}
- virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
+ VIRTIO_OPS(hw) = &legacy_ops;
dev->modern = false;
msix_detect:
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index c3db36d2fc..8b07c4a369 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -12,6 +12,8 @@
#include <rte_bus_pci.h>
#include <rte_ethdev_driver.h>
+#include "virtio.h"
+
struct virtqueue;
struct virtnet_ctl;
@@ -214,68 +216,6 @@ struct virtio_pci_common_cfg {
uint32_t queue_used_hi; /* read-write */
};
-struct virtio_hw;
-
-struct virtio_pci_ops {
- void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset,
- void *dst, int len);
- void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset,
- const void *src, int len);
-
- uint8_t (*get_status)(struct virtio_hw *hw);
- void (*set_status)(struct virtio_hw *hw, uint8_t status);
-
- uint64_t (*get_features)(struct virtio_hw *hw);
- void (*set_features)(struct virtio_hw *hw, uint64_t features);
- int (*features_ok)(struct virtio_hw *hw);
-
- uint8_t (*get_isr)(struct virtio_hw *hw);
-
- uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
-
- uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq,
- uint16_t vec);
-
- uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
- int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
- void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
- void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
- int (*dev_close)(struct virtio_hw *hw);
-};
-
-struct virtio_net_config;
-
-struct virtio_hw {
- struct virtqueue **vqs;
- uint64_t guest_features;
- uint16_t vtnet_hdr_size;
- uint8_t started;
- uint8_t weak_barriers;
- uint8_t vlan_strip;
- uint8_t has_tx_offload;
- uint8_t has_rx_offload;
- uint8_t use_vec_rx;
- uint8_t use_vec_tx;
- uint8_t use_inorder_rx;
- uint8_t use_inorder_tx;
- uint8_t opened;
- uint16_t port_id;
- uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
- uint32_t speed; /* link speed in MB */
- uint8_t duplex;
- uint8_t use_msix;
- uint16_t max_mtu;
- /*
- * App management thread and virtio interrupt handler thread
- * both can change device state, this lock is meant to avoid
- * such a contention.
- */
- rte_spinlock_t state_lock;
- struct rte_mbuf **inject_pkts;
- uint16_t max_queue_pairs;
- uint64_t req_guest_features;
- struct virtnet_ctl *cvq;
-};
struct virtio_pci_dev {
struct virtio_hw hw;
@@ -290,19 +230,6 @@ struct virtio_pci_dev {
#define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
-/*
- * While virtio_hw is stored in shared memory, this structure stores
- * some infos that may vary in the multiple process model locally.
- * For example, the vtpci_ops pointer.
- */
-struct virtio_hw_internal {
- const struct virtio_pci_ops *vtpci_ops;
-};
-
-#define VTPCI_OPS(hw) (virtio_hw_internal[(hw)->port_id].vtpci_ops)
-
-extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
-
/*
* This structure is just a reference to read
* net device specific config space; it just a chodu structure
@@ -380,8 +307,7 @@ enum virtio_msix_status vtpci_msix_detect(struct rte_pci_device *dev);
void vtpci_legacy_ioport_unmap(struct virtio_hw *hw);
int vtpci_legacy_ioport_map(struct virtio_hw *hw);
-extern const struct virtio_pci_ops legacy_ops;
-extern const struct virtio_pci_ops modern_ops;
-extern const struct virtio_pci_ops virtio_user_ops;
+extern const struct virtio_ops legacy_ops;
+extern const struct virtio_ops modern_ops;
#endif /* _VIRTIO_PCI_H_ */
diff --git a/drivers/net/virtio/virtio_pci_ethdev.c b/drivers/net/virtio/virtio_pci_ethdev.c
index 17342ae7d8..347dc291e8 100644
--- a/drivers/net/virtio/virtio_pci_ethdev.c
+++ b/drivers/net/virtio/virtio_pci_ethdev.c
@@ -19,6 +19,7 @@
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include "virtio.h"
#include "virtio_ethdev.h"
#include "virtio_pci.h"
#include "virtio_logs.h"
@@ -83,9 +84,9 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
}
} else {
if (dev->modern)
- VTPCI_OPS(hw) = &modern_ops;
+ VIRTIO_OPS(hw) = &modern_ops;
else
- VTPCI_OPS(hw) = &legacy_ops;
+ VIRTIO_OPS(hw) = &legacy_ops;
ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
if (ret < 0) {
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 1420db32be..14468ddf52 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -20,7 +20,7 @@
#include "virtio_ethdev.h"
#include "virtio_logs.h"
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtqueue.h"
#include "virtio_rxtx.h"
#include "virtio_user/virtio_user_dev.h"
@@ -478,7 +478,7 @@ virtio_user_dev_close(struct virtio_hw *hw)
return 0;
}
-const struct virtio_pci_ops virtio_user_ops = {
+const struct virtio_ops virtio_user_ops = {
.read_dev_cfg = virtio_user_read_dev_config,
.write_dev_cfg = virtio_user_write_dev_config,
.get_status = virtio_user_get_status,
@@ -635,7 +635,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
hw->port_id = data->port_id;
dev->port_id = data->port_id;
- virtio_hw_internal[hw->port_id].vtpci_ops = &virtio_user_ops;
+ VIRTIO_OPS(hw) = &virtio_user_ops;
/*
* MSIX is required to enable LSC (see virtio_init_device).
* Here just pretend that we support msix.
@@ -701,7 +701,7 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
dev = eth_dev->data->dev_private;
hw = &dev->hw;
- VTPCI_OPS(hw) = &virtio_user_ops;
+ VIRTIO_OPS(hw) = &virtio_user_ops;
if (eth_virtio_dev_init(eth_dev) < 0) {
PMD_INIT_LOG(ERR, "eth_virtio_dev_init fails");
diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
index e4a1393816..9d2089766b 100644
--- a/drivers/net/virtio/virtqueue.h
+++ b/drivers/net/virtio/virtqueue.h
@@ -564,7 +564,7 @@ virtqueue_kick_prepare_packed(struct virtqueue *vq)
static inline void
virtqueue_notify(struct virtqueue *vq)
{
- VTPCI_OPS(vq->hw)->notify_queue(vq->hw, vq);
+ VIRTIO_OPS(vq->hw)->notify_queue(vq->hw, vq);
}
#ifdef RTE_LIBRTE_VIRTIO_DEBUG_DUMP
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header Maxime Coquelin
@ 2020-12-30 3:09 ` Xia, Chenbo
2021-01-06 10:08 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:09 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 16/40] net/virtio: introduce generic virtio header
>
> This patch moves virtio_hw and virtio callbacks into
> a generic virtio header, now that they have been
> curated from PCI references.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.h | 75 ++++++++++++++++++++++
> drivers/net/virtio/virtio_ethdev.c | 20 +++---
> drivers/net/virtio/virtio_pci.c | 26 ++++----
> drivers/net/virtio/virtio_pci.h | 82 ++-----------------------
> drivers/net/virtio/virtio_pci_ethdev.c | 5 +-
> drivers/net/virtio/virtio_user_ethdev.c | 8 +--
> drivers/net/virtio/virtqueue.h | 2 +-
> 7 files changed, 110 insertions(+), 108 deletions(-)
> create mode 100644 drivers/net/virtio/virtio.h
>
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> new file mode 100644
> index 0000000000..eb078bc227
> --- /dev/null
> +++ b/drivers/net/virtio/virtio.h
> @@ -0,0 +1,75 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2020 Red Hat, Inc.
> + */
> +
> +#ifndef _VIRTIO_H_
> +#define _VIRTIO_H_
> +
> +#include <rte_ether.h>
> +
> +struct virtio_hw {
> + struct virtqueue **vqs;
> + uint64_t guest_features;
> + uint16_t vtnet_hdr_size;
> + uint8_t started;
> + uint8_t weak_barriers;
> + uint8_t vlan_strip;
> + uint8_t has_tx_offload;
> + uint8_t has_rx_offload;
> + uint8_t use_vec_rx;
> + uint8_t use_vec_tx;
> + uint8_t use_inorder_rx;
> + uint8_t use_inorder_tx;
> + uint8_t opened;
> + uint16_t port_id;
> + uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> + uint32_t speed; /* link speed in MB */
> + uint8_t duplex;
> + uint8_t use_msix;
> + uint16_t max_mtu;
> + /*
> + * App management thread and virtio interrupt handler thread
> + * both can change device state, this lock is meant to avoid
> + * such a contention.
> + */
> + rte_spinlock_t state_lock;
> + struct rte_mbuf **inject_pkts;
> + uint16_t max_queue_pairs;
> + uint64_t req_guest_features;
> + struct virtnet_ctl *cvq;
> +};
> +
> +struct virtio_ops {
> + void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset, void *dst, int
> len);
> + void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset, const void
> *src, int len);
> + uint8_t (*get_status)(struct virtio_hw *hw);
> + void (*set_status)(struct virtio_hw *hw, uint8_t status);
> + uint64_t (*get_features)(struct virtio_hw *hw);
> + void (*set_features)(struct virtio_hw *hw, uint64_t features);
> + int (*features_ok)(struct virtio_hw *hw);
> + uint8_t (*get_isr)(struct virtio_hw *hw);
> + uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
> + uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq,
> uint16_t vec);
> + uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
> + int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + int (*dev_close)(struct virtio_hw *hw);
> +};
> +
> +/*
> + * While virtio_hw is stored in shared memory, this structure stores
> + * some infos that may vary in the multiple process model locally.
> + * For example, the vtpci_ops pointer.
> + */
> +struct virtio_hw_internal {
> + const struct virtio_ops *virtio_ops;
> +};
> +
> +#define VIRTIO_OPS(hw) (virtio_hw_internal[(hw)->port_id].virtio_ops)
> +
> +extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
> +
> +
> +#endif /* _VIRTIO_H_ */
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 86d8930e78..80b3fdba9e 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -446,7 +446,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> * Read the virtqueue size from the Queue Size field
> * Always power of 2 and if 0 virtqueue does not exist
> */
> - vq_size = VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
> + vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
> PMD_INIT_LOG(DEBUG, "vq_size: %u", vq_size);
> if (vq_size == 0) {
> PMD_INIT_LOG(ERR, "virtqueue does not exist");
> @@ -608,7 +608,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> }
> }
>
> - if (VTPCI_OPS(hw)->setup_queue(hw, vq) < 0) {
> + if (VIRTIO_OPS(hw)->setup_queue(hw, vq) < 0) {
> PMD_INIT_LOG(ERR, "setup_queue failed");
> return -EINVAL;
> }
> @@ -703,7 +703,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
>
> /* reset the NIC */
> if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
> - VTPCI_OPS(hw)->set_config_irq(hw, VIRTIO_MSI_NO_VECTOR);
> + VIRTIO_OPS(hw)->set_config_irq(hw, VIRTIO_MSI_NO_VECTOR);
> if (intr_conf->rxq)
> virtio_queues_unbind_intr(dev);
>
> @@ -718,7 +718,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
> virtio_dev_free_mbufs(dev);
> virtio_free_queues(hw);
>
> - return VTPCI_OPS(hw)->dev_close(hw);
> + return VIRTIO_OPS(hw)->dev_close(hw);
> }
>
> static int
> @@ -1290,7 +1290,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t
> req_features)
> req_features);
>
> /* Read device(host) feature bits */
> - host_features = VTPCI_OPS(hw)->get_features(hw);
> + host_features = VIRTIO_OPS(hw)->get_features(hw);
> PMD_INIT_LOG(DEBUG, "host_features before negotiate = %" PRIx64,
> host_features);
>
> @@ -1315,7 +1315,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t
> req_features)
> PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
> hw->guest_features);
>
> - if (VTPCI_OPS(hw)->features_ok(hw) < 0) {
> + if (VIRTIO_OPS(hw)->features_ok(hw) < 0) {
> PMD_INIT_LOG(ERR, "Features not OK at bus level\n");
> return -1;
> }
> @@ -1551,7 +1551,7 @@ virtio_queues_bind_intr(struct rte_eth_dev *dev)
> PMD_INIT_LOG(INFO, "queue/interrupt binding");
> for (i = 0; i < dev->data->nb_rx_queues; ++i) {
> dev->intr_handle->intr_vec[i] = i + 1;
> - if (VTPCI_OPS(hw)->set_queue_irq(hw, hw->vqs[i * 2], i + 1) ==
> + if (VIRTIO_OPS(hw)->set_queue_irq(hw, hw->vqs[i * 2], i + 1) ==
> VIRTIO_MSI_NO_VECTOR) {
> PMD_DRV_LOG(ERR, "failed to set queue vector");
> return -EBUSY;
> @@ -1569,7 +1569,7 @@ virtio_queues_unbind_intr(struct rte_eth_dev *dev)
>
> PMD_INIT_LOG(INFO, "queue/interrupt unbinding");
> for (i = 0; i < dev->data->nb_rx_queues; ++i)
> - VTPCI_OPS(hw)->set_queue_irq(hw,
> + VIRTIO_OPS(hw)->set_queue_irq(hw,
> hw->vqs[i * VTNET_CQ],
> VIRTIO_MSI_NO_VECTOR);
> }
> @@ -2081,7 +2081,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>
> if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
> /* Enable vector (0) for Link State Intrerrupt */
> - if (VTPCI_OPS(hw)->set_config_irq(hw, 0) ==
> + if (VIRTIO_OPS(hw)->set_config_irq(hw, 0) ==
> VIRTIO_MSI_NO_VECTOR) {
> PMD_DRV_LOG(ERR, "failed to set config vector");
> return -EBUSY;
> @@ -2411,7 +2411,7 @@ virtio_dev_info_get(struct rte_eth_dev *dev, struct
> rte_eth_dev_info *dev_info)
> dev_info->max_rx_pktlen = VIRTIO_MAX_RX_PKTLEN;
> dev_info->max_mac_addrs = VIRTIO_MAX_MAC_ADDRS;
>
> - host_features = VTPCI_OPS(hw)->get_features(hw);
> + host_features = VIRTIO_OPS(hw)->get_features(hw);
> dev_info->rx_offload_capa = DEV_RX_OFFLOAD_VLAN_STRIP;
> dev_info->rx_offload_capa |= DEV_RX_OFFLOAD_JUMBO_FRAME;
> if (host_features & (1ULL << VIRTIO_NET_F_GUEST_CSUM)) {
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 230a438bf7..df69fcdd45 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -267,7 +267,7 @@ legacy_dev_close(struct virtio_hw *hw)
> return 0;
> }
>
> -const struct virtio_pci_ops legacy_ops = {
> +const struct virtio_ops legacy_ops = {
> .read_dev_cfg = legacy_read_dev_config,
> .write_dev_cfg = legacy_write_dev_config,
> .get_status = legacy_get_status,
> @@ -515,7 +515,7 @@ modern_dev_close(struct virtio_hw *hw)
> return 0;
> }
>
> -const struct virtio_pci_ops modern_ops = {
> +const struct virtio_ops modern_ops = {
> .read_dev_cfg = modern_read_dev_config,
> .write_dev_cfg = modern_write_dev_config,
> .get_status = modern_get_status,
> @@ -538,14 +538,14 @@ void
> vtpci_read_dev_config(struct virtio_hw *hw, size_t offset,
> void *dst, int length)
> {
> - VTPCI_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
> + VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
> }
>
> void
> vtpci_write_dev_config(struct virtio_hw *hw, size_t offset,
> const void *src, int length)
> {
> - VTPCI_OPS(hw)->write_dev_cfg(hw, offset, src, length);
> + VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
> }
>
> uint64_t
> @@ -558,7 +558,7 @@ vtpci_negotiate_features(struct virtio_hw *hw, uint64_t
> host_features)
> * host all support.
> */
> features = host_features & hw->guest_features;
> - VTPCI_OPS(hw)->set_features(hw, features);
> + VIRTIO_OPS(hw)->set_features(hw, features);
>
> return features;
> }
> @@ -566,9 +566,9 @@ vtpci_negotiate_features(struct virtio_hw *hw, uint64_t
> host_features)
> void
> vtpci_reset(struct virtio_hw *hw)
> {
> - VTPCI_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
> + VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
> /* flush status write */
> - VTPCI_OPS(hw)->get_status(hw);
> + VIRTIO_OPS(hw)->get_status(hw);
> }
>
> void
> @@ -581,21 +581,21 @@ void
> vtpci_set_status(struct virtio_hw *hw, uint8_t status)
> {
> if (status != VIRTIO_CONFIG_STATUS_RESET)
> - status |= VTPCI_OPS(hw)->get_status(hw);
> + status |= VIRTIO_OPS(hw)->get_status(hw);
>
> - VTPCI_OPS(hw)->set_status(hw, status);
> + VIRTIO_OPS(hw)->set_status(hw, status);
> }
>
> uint8_t
> vtpci_get_status(struct virtio_hw *hw)
> {
> - return VTPCI_OPS(hw)->get_status(hw);
> + return VIRTIO_OPS(hw)->get_status(hw);
> }
>
> uint8_t
> vtpci_isr(struct virtio_hw *hw)
> {
> - return VTPCI_OPS(hw)->get_isr(hw);
> + return VIRTIO_OPS(hw)->get_isr(hw);
> }
>
> static void *
> @@ -772,7 +772,7 @@ vtpci_init(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> */
> if (virtio_read_caps(pci_dev, hw) == 0) {
> PMD_INIT_LOG(INFO, "modern virtio pci detected.");
> - virtio_hw_internal[hw->port_id].vtpci_ops = &modern_ops;
> + VIRTIO_OPS(hw) = &modern_ops;
> dev->modern = true;
> goto msix_detect;
> }
> @@ -791,7 +791,7 @@ vtpci_init(struct rte_pci_device *pci_dev, struct
> virtio_pci_dev *dev)
> return -1;
> }
>
> - virtio_hw_internal[hw->port_id].vtpci_ops = &legacy_ops;
> + VIRTIO_OPS(hw) = &legacy_ops;
> dev->modern = false;
>
> msix_detect:
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index c3db36d2fc..8b07c4a369 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -12,6 +12,8 @@
> #include <rte_bus_pci.h>
> #include <rte_ethdev_driver.h>
>
> +#include "virtio.h"
> +
> struct virtqueue;
> struct virtnet_ctl;
>
> @@ -214,68 +216,6 @@ struct virtio_pci_common_cfg {
> uint32_t queue_used_hi; /* read-write */
> };
>
> -struct virtio_hw;
> -
> -struct virtio_pci_ops {
> - void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset,
> - void *dst, int len);
> - void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset,
> - const void *src, int len);
> -
> - uint8_t (*get_status)(struct virtio_hw *hw);
> - void (*set_status)(struct virtio_hw *hw, uint8_t status);
> -
> - uint64_t (*get_features)(struct virtio_hw *hw);
> - void (*set_features)(struct virtio_hw *hw, uint64_t features);
> - int (*features_ok)(struct virtio_hw *hw);
> -
> - uint8_t (*get_isr)(struct virtio_hw *hw);
> -
> - uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
> -
> - uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq,
> - uint16_t vec);
> -
> - uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
> - int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> - void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> - void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> - int (*dev_close)(struct virtio_hw *hw);
> -};
> -
> -struct virtio_net_config;
> -
> -struct virtio_hw {
> - struct virtqueue **vqs;
> - uint64_t guest_features;
> - uint16_t vtnet_hdr_size;
> - uint8_t started;
> - uint8_t weak_barriers;
> - uint8_t vlan_strip;
> - uint8_t has_tx_offload;
> - uint8_t has_rx_offload;
> - uint8_t use_vec_rx;
> - uint8_t use_vec_tx;
> - uint8_t use_inorder_rx;
> - uint8_t use_inorder_tx;
> - uint8_t opened;
> - uint16_t port_id;
> - uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> - uint32_t speed; /* link speed in MB */
> - uint8_t duplex;
> - uint8_t use_msix;
> - uint16_t max_mtu;
> - /*
> - * App management thread and virtio interrupt handler thread
> - * both can change device state, this lock is meant to avoid
> - * such a contention.
> - */
> - rte_spinlock_t state_lock;
> - struct rte_mbuf **inject_pkts;
> - uint16_t max_queue_pairs;
> - uint64_t req_guest_features;
> - struct virtnet_ctl *cvq;
> -};
>
> struct virtio_pci_dev {
> struct virtio_hw hw;
> @@ -290,19 +230,6 @@ struct virtio_pci_dev {
>
> #define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
>
> -/*
> - * While virtio_hw is stored in shared memory, this structure stores
> - * some infos that may vary in the multiple process model locally.
> - * For example, the vtpci_ops pointer.
> - */
> -struct virtio_hw_internal {
> - const struct virtio_pci_ops *vtpci_ops;
> -};
> -
> -#define VTPCI_OPS(hw) (virtio_hw_internal[(hw)->port_id].vtpci_ops)
> -
> -extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
> -
> /*
> * This structure is just a reference to read
> * net device specific config space; it just a chodu structure
> @@ -380,8 +307,7 @@ enum virtio_msix_status vtpci_msix_detect(struct
> rte_pci_device *dev);
> void vtpci_legacy_ioport_unmap(struct virtio_hw *hw);
> int vtpci_legacy_ioport_map(struct virtio_hw *hw);
>
> -extern const struct virtio_pci_ops legacy_ops;
> -extern const struct virtio_pci_ops modern_ops;
> -extern const struct virtio_pci_ops virtio_user_ops;
> +extern const struct virtio_ops legacy_ops;
> +extern const struct virtio_ops modern_ops;
>
> #endif /* _VIRTIO_PCI_H_ */
> diff --git a/drivers/net/virtio/virtio_pci_ethdev.c
> b/drivers/net/virtio/virtio_pci_ethdev.c
> index 17342ae7d8..347dc291e8 100644
> --- a/drivers/net/virtio/virtio_pci_ethdev.c
> +++ b/drivers/net/virtio/virtio_pci_ethdev.c
> @@ -19,6 +19,7 @@
> #include <rte_dev.h>
> #include <rte_kvargs.h>
>
> +#include "virtio.h"
> #include "virtio_ethdev.h"
> #include "virtio_pci.h"
> #include "virtio_logs.h"
> @@ -83,9 +84,9 @@ eth_virtio_pci_init(struct rte_eth_dev *eth_dev)
> }
> } else {
> if (dev->modern)
> - VTPCI_OPS(hw) = &modern_ops;
> + VIRTIO_OPS(hw) = &modern_ops;
> else
> - VTPCI_OPS(hw) = &legacy_ops;
> + VIRTIO_OPS(hw) = &legacy_ops;
>
> ret = virtio_remap_pci(RTE_ETH_DEV_TO_PCI(eth_dev), dev);
> if (ret < 0) {
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 1420db32be..14468ddf52 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -20,7 +20,7 @@
>
> #include "virtio_ethdev.h"
> #include "virtio_logs.h"
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtqueue.h"
> #include "virtio_rxtx.h"
> #include "virtio_user/virtio_user_dev.h"
> @@ -478,7 +478,7 @@ virtio_user_dev_close(struct virtio_hw *hw)
> return 0;
> }
>
> -const struct virtio_pci_ops virtio_user_ops = {
> +const struct virtio_ops virtio_user_ops = {
> .read_dev_cfg = virtio_user_read_dev_config,
> .write_dev_cfg = virtio_user_write_dev_config,
> .get_status = virtio_user_get_status,
> @@ -635,7 +635,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
>
> hw->port_id = data->port_id;
> dev->port_id = data->port_id;
> - virtio_hw_internal[hw->port_id].vtpci_ops = &virtio_user_ops;
> + VIRTIO_OPS(hw) = &virtio_user_ops;
> /*
> * MSIX is required to enable LSC (see virtio_init_device).
> * Here just pretend that we support msix.
> @@ -701,7 +701,7 @@ virtio_user_pmd_probe(struct rte_vdev_device *vdev)
>
> dev = eth_dev->data->dev_private;
> hw = &dev->hw;
> - VTPCI_OPS(hw) = &virtio_user_ops;
> + VIRTIO_OPS(hw) = &virtio_user_ops;
>
> if (eth_virtio_dev_init(eth_dev) < 0) {
> PMD_INIT_LOG(ERR, "eth_virtio_dev_init fails");
> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
> index e4a1393816..9d2089766b 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -564,7 +564,7 @@ virtqueue_kick_prepare_packed(struct virtqueue *vq)
> static inline void
> virtqueue_notify(struct virtqueue *vq)
> {
> - VTPCI_OPS(vq->hw)->notify_queue(vq->hw, vq);
> + VIRTIO_OPS(vq->hw)->notify_queue(vq->hw, vq);
> }
>
> #ifdef RTE_LIBRTE_VIRTIO_DEBUG_DUMP
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header Maxime Coquelin
2020-12-30 3:09 ` Xia, Chenbo
@ 2021-01-06 10:08 ` David Marchand
2021-01-15 9:39 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 10:08 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> new file mode 100644
> index 0000000000..eb078bc227
> --- /dev/null
> +++ b/drivers/net/virtio/virtio.h
> @@ -0,0 +1,75 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2020 Red Hat, Inc.
2021?
> + */
> +
> +#ifndef _VIRTIO_H_
> +#define _VIRTIO_H_
> +
> +#include <rte_ether.h>
> +
> +struct virtio_hw {
> + struct virtqueue **vqs;
> + uint64_t guest_features;
> + uint16_t vtnet_hdr_size;
> + uint8_t started;
> + uint8_t weak_barriers;
> + uint8_t vlan_strip;
> + uint8_t has_tx_offload;
> + uint8_t has_rx_offload;
> + uint8_t use_vec_rx;
> + uint8_t use_vec_tx;
> + uint8_t use_inorder_rx;
> + uint8_t use_inorder_tx;
> + uint8_t opened;
> + uint16_t port_id;
> + uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> + uint32_t speed; /* link speed in MB */
> + uint8_t duplex;
> + uint8_t use_msix;
> + uint16_t max_mtu;
> + /*
> + * App management thread and virtio interrupt handler thread
> + * both can change device state, this lock is meant to avoid
> + * such a contention.
> + */
> + rte_spinlock_t state_lock;
> + struct rte_mbuf **inject_pkts;
> + uint16_t max_queue_pairs;
> + uint64_t req_guest_features;
> + struct virtnet_ctl *cvq;
> +};
> +
> +struct virtio_ops {
> + void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset, void *dst, int len);
> + void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset, const void *src, int len);
> + uint8_t (*get_status)(struct virtio_hw *hw);
> + void (*set_status)(struct virtio_hw *hw, uint8_t status);
> + uint64_t (*get_features)(struct virtio_hw *hw);
> + void (*set_features)(struct virtio_hw *hw, uint64_t features);
> + int (*features_ok)(struct virtio_hw *hw);
> + uint8_t (*get_isr)(struct virtio_hw *hw);
> + uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
> + uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq, uint16_t vec);
> + uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
> + int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
> + int (*dev_close)(struct virtio_hw *hw);
> +};
> +
> +/*
> + * While virtio_hw is stored in shared memory, this structure stores
> + * some infos that may vary in the multiple process model locally.
> + * For example, the vtpci_ops pointer.
You can remove this comment on vtpci_ops.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header
2021-01-06 10:08 ` David Marchand
@ 2021-01-15 9:39 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 9:39 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 11:08 AM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
>> new file mode 100644
>> index 0000000000..eb078bc227
>> --- /dev/null
>> +++ b/drivers/net/virtio/virtio.h
>> @@ -0,0 +1,75 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2010-2014 Intel Corporation
>> + * Copyright(c) 2020 Red Hat, Inc.
>
> 2021?
Well, it was still 2020 when I submitted the patch :)
I'll change it when fixing comment below
>
>
>> + */
>> +
>> +#ifndef _VIRTIO_H_
>> +#define _VIRTIO_H_
>> +
>> +#include <rte_ether.h>
>> +
>> +struct virtio_hw {
>> + struct virtqueue **vqs;
>> + uint64_t guest_features;
>> + uint16_t vtnet_hdr_size;
>> + uint8_t started;
>> + uint8_t weak_barriers;
>> + uint8_t vlan_strip;
>> + uint8_t has_tx_offload;
>> + uint8_t has_rx_offload;
>> + uint8_t use_vec_rx;
>> + uint8_t use_vec_tx;
>> + uint8_t use_inorder_rx;
>> + uint8_t use_inorder_tx;
>> + uint8_t opened;
>> + uint16_t port_id;
>> + uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
>> + uint32_t speed; /* link speed in MB */
>> + uint8_t duplex;
>> + uint8_t use_msix;
>> + uint16_t max_mtu;
>> + /*
>> + * App management thread and virtio interrupt handler thread
>> + * both can change device state, this lock is meant to avoid
>> + * such a contention.
>> + */
>> + rte_spinlock_t state_lock;
>> + struct rte_mbuf **inject_pkts;
>> + uint16_t max_queue_pairs;
>> + uint64_t req_guest_features;
>> + struct virtnet_ctl *cvq;
>> +};
>> +
>> +struct virtio_ops {
>> + void (*read_dev_cfg)(struct virtio_hw *hw, size_t offset, void *dst, int len);
>> + void (*write_dev_cfg)(struct virtio_hw *hw, size_t offset, const void *src, int len);
>> + uint8_t (*get_status)(struct virtio_hw *hw);
>> + void (*set_status)(struct virtio_hw *hw, uint8_t status);
>> + uint64_t (*get_features)(struct virtio_hw *hw);
>> + void (*set_features)(struct virtio_hw *hw, uint64_t features);
>> + int (*features_ok)(struct virtio_hw *hw);
>> + uint8_t (*get_isr)(struct virtio_hw *hw);
>> + uint16_t (*set_config_irq)(struct virtio_hw *hw, uint16_t vec);
>> + uint16_t (*set_queue_irq)(struct virtio_hw *hw, struct virtqueue *vq, uint16_t vec);
>> + uint16_t (*get_queue_num)(struct virtio_hw *hw, uint16_t queue_id);
>> + int (*setup_queue)(struct virtio_hw *hw, struct virtqueue *vq);
>> + void (*del_queue)(struct virtio_hw *hw, struct virtqueue *vq);
>> + void (*notify_queue)(struct virtio_hw *hw, struct virtqueue *vq);
>> + int (*dev_close)(struct virtio_hw *hw);
>> +};
>> +
>> +/*
>> + * While virtio_hw is stored in shared memory, this structure stores
>> + * some infos that may vary in the multiple process model locally.
>> + * For example, the vtpci_ops pointer.
>
> You can remove this comment on vtpci_ops.
>
I think the comment is still valid, just that it now applied to
virtio_ops instead of vtpci_ops.
I'll fix it.
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (15 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 16/40] net/virtio: introduce generic virtio header Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:14 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in " Maxime Coquelin
` (23 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves all the Virtio definition to the generic
header. It also renames some helpers to no more reference
PCI.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/meson.build | 3 +-
drivers/net/virtio/virtio.c | 22 ++++
drivers/net/virtio/virtio.h | 94 +++++++++++++++
drivers/net/virtio/virtio_ethdev.c | 110 +++++++++---------
drivers/net/virtio/virtio_pci.c | 21 +---
drivers/net/virtio/virtio_pci.h | 90 --------------
drivers/net/virtio/virtio_ring.h | 2 +-
drivers/net/virtio/virtio_rxtx.c | 38 +++---
drivers/net/virtio/virtio_rxtx_packed_avx.c | 6 +-
.../net/virtio/virtio_user/vhost_kernel_tap.c | 2 +-
drivers/net/virtio/virtio_user_ethdev.c | 6 +-
drivers/net/virtio/virtqueue.c | 4 +-
drivers/net/virtio/virtqueue.h | 8 +-
13 files changed, 209 insertions(+), 197 deletions(-)
create mode 100644 drivers/net/virtio/virtio.c
diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
index 0b62418f33..7de41cd04d 100644
--- a/drivers/net/virtio/meson.build
+++ b/drivers/net/virtio/meson.build
@@ -1,7 +1,8 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
-sources += files('virtio_ethdev.c',
+sources += files('virtio.c',
+ 'virtio_ethdev.c',
'virtio_pci_ethdev.c',
'virtio_pci.c',
'virtio_rxtx.c',
diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
new file mode 100644
index 0000000000..d8d6bf7add
--- /dev/null
+++ b/drivers/net/virtio/virtio.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2020 Red Hat, Inc.
+ */
+
+#include "virtio.h"
+
+uint64_t
+virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
+{
+ uint64_t features;
+
+ /*
+ * Limit negotiated features to what the driver, virtqueue, and
+ * host all support.
+ */
+ features = host_features & hw->guest_features;
+ VIRTIO_OPS(hw)->set_features(hw, features);
+
+ return features;
+}
+
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
index eb078bc227..9e787803a4 100644
--- a/drivers/net/virtio/virtio.h
+++ b/drivers/net/virtio/virtio.h
@@ -8,6 +8,86 @@
#include <rte_ether.h>
+/* The feature bitmap for virtio net */
+#define VIRTIO_NET_F_CSUM 0 /* Host handles pkts w/ partial csum */
+#define VIRTIO_NET_F_GUEST_CSUM 1 /* Guest handles pkts w/ partial csum */
+#define VIRTIO_NET_F_MTU 3 /* Initial MTU advice. */
+#define VIRTIO_NET_F_MAC 5 /* Host has given MAC address. */
+#define VIRTIO_NET_F_GUEST_TSO4 7 /* Guest can handle TSOv4 in. */
+#define VIRTIO_NET_F_GUEST_TSO6 8 /* Guest can handle TSOv6 in. */
+#define VIRTIO_NET_F_GUEST_ECN 9 /* Guest can handle TSO[6] w/ ECN in. */
+#define VIRTIO_NET_F_GUEST_UFO 10 /* Guest can handle UFO in. */
+#define VIRTIO_NET_F_HOST_TSO4 11 /* Host can handle TSOv4 in. */
+#define VIRTIO_NET_F_HOST_TSO6 12 /* Host can handle TSOv6 in. */
+#define VIRTIO_NET_F_HOST_ECN 13 /* Host can handle TSO[6] w/ ECN in. */
+#define VIRTIO_NET_F_HOST_UFO 14 /* Host can handle UFO in. */
+#define VIRTIO_NET_F_MRG_RXBUF 15 /* Host can merge receive buffers. */
+#define VIRTIO_NET_F_STATUS 16 /* virtio_net_config.status available */
+#define VIRTIO_NET_F_CTRL_VQ 17 /* Control channel available */
+#define VIRTIO_NET_F_CTRL_RX 18 /* Control channel RX mode support */
+#define VIRTIO_NET_F_CTRL_VLAN 19 /* Control channel VLAN filtering */
+#define VIRTIO_NET_F_CTRL_RX_EXTRA 20 /* Extra RX mode control support */
+#define VIRTIO_NET_F_GUEST_ANNOUNCE 21 /* Guest can announce device on the network */
+#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow Steering */
+#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
+
+/*
+ * Do we get callbacks when the ring is completely used,
+ * even if we've suppressed them?
+ */
+#define VIRTIO_F_NOTIFY_ON_EMPTY 24
+
+/* Can the device handle any descriptor layout? */
+#define VIRTIO_F_ANY_LAYOUT 27
+
+/* We support indirect buffer descriptors */
+#define VIRTIO_RING_F_INDIRECT_DESC 28
+
+#define VIRTIO_F_VERSION_1 32
+#define VIRTIO_F_IOMMU_PLATFORM 33
+#define VIRTIO_F_RING_PACKED 34
+
+/*
+ * Some VirtIO feature bits (currently bits 28 through 31) are
+ * reserved for the transport being used (eg. virtio_ring), the
+ * rest are per-device feature bits.
+ */
+#define VIRTIO_TRANSPORT_F_START 28
+#define VIRTIO_TRANSPORT_F_END 34
+
+/*
+ * Inorder feature indicates that all buffers are used by the device
+ * in the same order in which they have been made available.
+ */
+#define VIRTIO_F_IN_ORDER 35
+
+/*
+ * This feature indicates that memory accesses by the driver and the device
+ * are ordered in a way described by the platform.
+ */
+#define VIRTIO_F_ORDER_PLATFORM 36
+
+/*
+ * This feature indicates that the driver passes extra data (besides
+ * identifying the virtqueue) in its device notifications.
+ */
+#define VIRTIO_F_NOTIFICATION_DATA 38
+
+/* Device set linkspeed and duplex */
+#define VIRTIO_NET_F_SPEED_DUPLEX 63
+
+/*
+ * The Guest publishes the used index for which it expects an interrupt
+ * at the end of the avail ring. Host should ignore the avail->flags field
+ *
+ * The Host publishes the avail index for which it expects a kick
+ * at the end of the used ring. Guest should ignore the used->flags field.
+ */
+#define VIRTIO_RING_F_EVENT_IDX 29
+
+#define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
+#define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
+
struct virtio_hw {
struct virtqueue **vqs;
uint64_t guest_features;
@@ -72,4 +152,18 @@ struct virtio_hw_internal {
extern struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS];
+static inline int
+virtio_with_feature(struct virtio_hw *hw, uint64_t bit)
+{
+ return (hw->guest_features & (1ULL << bit)) != 0;
+}
+
+static inline int
+virtio_with_packed_queue(struct virtio_hw *hw)
+{
+ return virtio_with_feature(hw, VIRTIO_F_RING_PACKED);
+}
+
+uint64_t virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features);
+
#endif /* _VIRTIO_H_ */
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 80b3fdba9e..c9085f2f35 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -339,7 +339,7 @@ virtio_send_command(struct virtnet_ctl *cvq, struct virtio_pmd_ctrl *ctrl,
memcpy(cvq->virtio_net_hdr_mz->addr, ctrl,
sizeof(struct virtio_pmd_ctrl));
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
result = virtio_send_command_packed(cvq, ctrl, dlen, pkt_num);
else
result = virtio_send_command_split(cvq, ctrl, dlen, pkt_num);
@@ -383,7 +383,7 @@ virtio_get_nr_vq(struct virtio_hw *hw)
{
uint16_t nr_vq = hw->max_queue_pairs * 2;
- if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
+ if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
nr_vq += 1;
return nr_vq;
@@ -405,7 +405,7 @@ virtio_init_vring(struct virtqueue *vq)
vq->vq_desc_tail_idx = (uint16_t)(vq->vq_nentries - 1);
vq->vq_free_cnt = vq->vq_nentries;
memset(vq->vq_descx, 0, sizeof(struct vq_desc_extra) * vq->vq_nentries);
- if (vtpci_packed_queue(vq->hw)) {
+ if (virtio_with_packed_queue(vq->hw)) {
vring_init_packed(&vq->vq_packed.ring, ring_mem,
VIRTIO_PCI_VRING_ALIGN, size);
vring_desc_init_packed(vq, size);
@@ -453,7 +453,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
return -EINVAL;
}
- if (!vtpci_packed_queue(hw) && !rte_is_power_of_2(vq_size)) {
+ if (!virtio_with_packed_queue(hw) && !rte_is_power_of_2(vq_size)) {
PMD_INIT_LOG(ERR, "split virtqueue size is not power of 2");
return -EINVAL;
}
@@ -486,7 +486,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
vq->hw = hw;
vq->vq_queue_index = vtpci_queue_idx;
vq->vq_nentries = vq_size;
- if (vtpci_packed_queue(hw)) {
+ if (virtio_with_packed_queue(hw)) {
vq->vq_packed.used_wrap_counter = 1;
vq->vq_packed.cached_flags = VRING_PACKED_DESC_F_AVAIL;
vq->vq_packed.event_flags_shadow = 0;
@@ -584,7 +584,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
memset(txr, 0, vq_size * sizeof(*txr));
for (i = 0; i < vq_size; i++) {
/* first indirect descriptor is always the tx header */
- if (!vtpci_packed_queue(hw)) {
+ if (!virtio_with_packed_queue(hw)) {
struct vring_desc *start_dp = txr[i].tx_indir;
vring_desc_init_split(start_dp,
RTE_DIM(txr[i].tx_indir));
@@ -729,7 +729,7 @@ virtio_dev_promiscuous_enable(struct rte_eth_dev *dev)
int dlen[1];
int ret;
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
PMD_INIT_LOG(INFO, "host does not support rx control");
return -ENOTSUP;
}
@@ -756,7 +756,7 @@ virtio_dev_promiscuous_disable(struct rte_eth_dev *dev)
int dlen[1];
int ret;
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
PMD_INIT_LOG(INFO, "host does not support rx control");
return -ENOTSUP;
}
@@ -783,7 +783,7 @@ virtio_dev_allmulticast_enable(struct rte_eth_dev *dev)
int dlen[1];
int ret;
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
PMD_INIT_LOG(INFO, "host does not support rx control");
return -ENOTSUP;
}
@@ -810,7 +810,7 @@ virtio_dev_allmulticast_disable(struct rte_eth_dev *dev)
int dlen[1];
int ret;
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_RX)) {
PMD_INIT_LOG(INFO, "host does not support rx control");
return -ENOTSUP;
}
@@ -1104,7 +1104,7 @@ virtio_set_hwaddr(struct virtio_hw *hw)
static void
virtio_get_hwaddr(struct virtio_hw *hw)
{
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MAC)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MAC)) {
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, mac),
&hw->mac_addr, RTE_ETHER_ADDR_LEN);
@@ -1122,7 +1122,7 @@ virtio_mac_table_set(struct virtio_hw *hw,
struct virtio_pmd_ctrl ctrl;
int err, len[2];
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_MAC_ADDR)) {
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_MAC_ADDR)) {
PMD_DRV_LOG(INFO, "host does not support mac table");
return -1;
}
@@ -1217,7 +1217,7 @@ virtio_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr)
memcpy(hw->mac_addr, mac_addr, RTE_ETHER_ADDR_LEN);
/* Use atomic update if available */
- if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_MAC_ADDR)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_MAC_ADDR)) {
struct virtio_pmd_ctrl ctrl;
int len = RTE_ETHER_ADDR_LEN;
@@ -1228,7 +1228,7 @@ virtio_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr)
return virtio_send_command(hw->cvq, &ctrl, &len, 1);
}
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_MAC))
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_MAC))
return -ENOTSUP;
virtio_set_hwaddr(hw);
@@ -1242,7 +1242,7 @@ virtio_vlan_filter_set(struct rte_eth_dev *dev, uint16_t vlan_id, int on)
struct virtio_pmd_ctrl ctrl;
int len;
- if (!vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN))
+ if (!virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN))
return -ENOTSUP;
ctrl.hdr.class = VIRTIO_NET_CTRL_VLAN;
@@ -1281,7 +1281,7 @@ virtio_intr_disable(struct rte_eth_dev *dev)
}
static int
-virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
+virtio_ethdev_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
{
uint64_t host_features;
@@ -1311,7 +1311,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
* guest feature bits.
*/
hw->guest_features = req_features;
- hw->guest_features = vtpci_negotiate_features(hw, host_features);
+ hw->guest_features = virtio_negotiate_features(hw, host_features);
PMD_INIT_LOG(DEBUG, "features after negotiate = %" PRIx64,
hw->guest_features);
@@ -1320,7 +1320,7 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
return -1;
}
- if (vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
+ if (virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
@@ -1454,7 +1454,7 @@ virtio_interrupt_handler(void *param)
RTE_ETH_EVENT_INTR_LSC,
NULL);
- if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
&status, sizeof(status));
@@ -1474,7 +1474,7 @@ set_rxtx_funcs(struct rte_eth_dev *eth_dev)
struct virtio_hw *hw = eth_dev->data->dev_private;
eth_dev->tx_pkt_prepare = virtio_xmit_pkts_prepare;
- if (vtpci_packed_queue(hw)) {
+ if (virtio_with_packed_queue(hw)) {
PMD_INIT_LOG(INFO,
"virtio: using packed ring %s Tx path on port %u",
hw->use_vec_tx ? "vectorized" : "standard",
@@ -1495,14 +1495,14 @@ set_rxtx_funcs(struct rte_eth_dev *eth_dev)
}
}
- if (vtpci_packed_queue(hw)) {
+ if (virtio_with_packed_queue(hw)) {
if (hw->use_vec_rx) {
PMD_INIT_LOG(INFO,
"virtio: using packed ring vectorized Rx path on port %u",
eth_dev->data->port_id);
eth_dev->rx_pkt_burst =
&virtio_recv_pkts_packed_vec;
- } else if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
+ } else if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
PMD_INIT_LOG(INFO,
"virtio: using packed ring mergeable buffer Rx path on port %u",
eth_dev->data->port_id);
@@ -1524,7 +1524,7 @@ set_rxtx_funcs(struct rte_eth_dev *eth_dev)
"virtio: using inorder Rx path on port %u",
eth_dev->data->port_id);
eth_dev->rx_pkt_burst = &virtio_recv_pkts_inorder;
- } else if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
+ } else if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
PMD_INIT_LOG(INFO,
"virtio: using mergeable buffer Rx path on port %u",
eth_dev->data->port_id);
@@ -1649,13 +1649,13 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
/* Tell the host we've known how to drive the device. */
vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
- if (virtio_negotiate_features(hw, req_features) < 0)
+ if (virtio_ethdev_negotiate_features(hw, req_features) < 0)
return -1;
- hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
+ hw->weak_barriers = !virtio_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
/* If host does not support both status and MSI-X then disable LSC */
- if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS) &&
+ if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS) &&
hw->use_msix != VIRTIO_MSIX_NONE)
eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
else
@@ -1664,9 +1664,9 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
/* Setting up rx_header size for the device */
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
- vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
- vtpci_with_feature(hw, VIRTIO_F_RING_PACKED))
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
+ virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
+ virtio_with_feature(hw, VIRTIO_F_RING_PACKED))
hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
else
hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr);
@@ -1681,7 +1681,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->mac_addr[3], hw->mac_addr[4], hw->mac_addr[5]);
if (hw->speed == ETH_SPEED_NUM_UNKNOWN) {
- if (vtpci_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
config = &local_config;
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, speed),
@@ -1697,14 +1697,14 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->duplex = ETH_LINK_FULL_DUPLEX;
PMD_INIT_LOG(DEBUG, "link speed = %d, duplex = %d",
hw->speed, hw->duplex);
- if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
config = &local_config;
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, mac),
&config->mac, sizeof(config->mac));
- if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
&config->status, sizeof(config->status));
@@ -1714,7 +1714,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
config->status = 0;
}
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MQ)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MQ)) {
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, max_virtqueue_pairs),
&config->max_virtqueue_pairs,
@@ -1727,7 +1727,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->max_queue_pairs = config->max_virtqueue_pairs;
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MTU)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MTU)) {
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, mtu),
&config->mtu,
@@ -1838,7 +1838,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
goto err_virtio_init;
if (vectorized) {
- if (!vtpci_packed_queue(hw)) {
+ if (!virtio_with_packed_queue(hw)) {
hw->use_vec_rx = 1;
} else {
#if !defined(CC_AVX512_SUPPORT)
@@ -1965,17 +1965,17 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs, uint32_t *speed, int *vect
static bool
rx_offload_enabled(struct virtio_hw *hw)
{
- return vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
- vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
- vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
+ return virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
+ virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
+ virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
}
static bool
tx_offload_enabled(struct virtio_hw *hw)
{
- return vtpci_with_feature(hw, VIRTIO_NET_F_CSUM) ||
- vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
- vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
+ return virtio_with_feature(hw, VIRTIO_NET_F_CSUM) ||
+ virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
+ virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
}
/*
@@ -2048,29 +2048,29 @@ virtio_dev_configure(struct rte_eth_dev *dev)
if ((rx_offloads & (DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM)) &&
- !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
+ !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
PMD_DRV_LOG(ERR,
"rx checksum not available on this host");
return -ENOTSUP;
}
if ((rx_offloads & DEV_RX_OFFLOAD_TCP_LRO) &&
- (!vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
- !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
+ (!virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
+ !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
PMD_DRV_LOG(ERR,
"Large Receive Offload not available on this host");
return -ENOTSUP;
}
/* start control queue */
- if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
+ if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
virtio_dev_cq_start(dev);
if (rx_offloads & DEV_RX_OFFLOAD_VLAN_STRIP)
hw->vlan_strip = 1;
- if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER)
- && !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
+ if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
+ !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
PMD_DRV_LOG(ERR,
"vlan filtering not available on this host");
return -ENOTSUP;
@@ -2087,12 +2087,12 @@ virtio_dev_configure(struct rte_eth_dev *dev)
return -EBUSY;
}
- if (vtpci_packed_queue(hw)) {
+ if (virtio_with_packed_queue(hw)) {
#if defined(RTE_ARCH_X86_64) && defined(CC_AVX512_SUPPORT)
if ((hw->use_vec_rx || hw->use_vec_tx) &&
(!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) ||
- !vtpci_with_feature(hw, VIRTIO_F_IN_ORDER) ||
- !vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
+ !virtio_with_feature(hw, VIRTIO_F_IN_ORDER) ||
+ !virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_512)) {
PMD_DRV_LOG(INFO,
"disabled packed ring vectorized path for requirements not met");
@@ -2105,7 +2105,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
#endif
if (hw->use_vec_rx) {
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
PMD_DRV_LOG(INFO,
"disabled packed ring vectorized rx for mrg_rxbuf enabled");
hw->use_vec_rx = 0;
@@ -2118,7 +2118,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
}
}
} else {
- if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) {
+ if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER)) {
hw->use_inorder_tx = 1;
hw->use_inorder_rx = 1;
hw->use_vec_rx = 0;
@@ -2132,7 +2132,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
hw->use_vec_rx = 0;
}
#endif
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
PMD_DRV_LOG(INFO,
"disabled split ring vectorized rx for mrg_rxbuf enabled");
hw->use_vec_rx = 0;
@@ -2350,7 +2350,7 @@ virtio_dev_link_update(struct rte_eth_dev *dev, __rte_unused int wait_to_complet
if (!hw->started) {
link.link_status = ETH_LINK_DOWN;
link.link_speed = ETH_SPEED_NUM_NONE;
- } else if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
+ } else if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
PMD_INIT_LOG(DEBUG, "Get link status from hw");
vtpci_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
@@ -2381,7 +2381,7 @@ virtio_dev_vlan_offload_set(struct rte_eth_dev *dev, int mask)
if (mask & ETH_VLAN_FILTER_MASK) {
if ((offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
- !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
+ !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
PMD_DRV_LOG(NOTICE,
"vlan filtering not available on this host");
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index df69fcdd45..9c07ebad00 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -356,7 +356,7 @@ modern_set_features(struct virtio_hw *hw, uint64_t features)
static int
modern_features_ok(struct virtio_hw *hw)
{
- if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
+ if (!virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
return -1;
}
@@ -479,12 +479,12 @@ modern_notify_queue(struct virtio_hw *hw, struct virtqueue *vq)
{
uint32_t notify_data;
- if (!vtpci_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
+ if (!virtio_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
rte_write16(vq->vq_queue_index, vq->notify_addr);
return;
}
- if (vtpci_with_feature(hw, VIRTIO_F_RING_PACKED)) {
+ if (virtio_with_feature(hw, VIRTIO_F_RING_PACKED)) {
/*
* Bit[0:15]: vq queue index
* Bit[16:30]: avail index
@@ -548,21 +548,6 @@ vtpci_write_dev_config(struct virtio_hw *hw, size_t offset,
VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
}
-uint64_t
-vtpci_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
-{
- uint64_t features;
-
- /*
- * Limit negotiated features to what the driver, virtqueue, and
- * host all support.
- */
- features = host_features & hw->guest_features;
- VIRTIO_OPS(hw)->set_features(hw, features);
-
- return features;
-}
-
void
vtpci_reset(struct virtio_hw *hw)
{
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 8b07c4a369..b02e5c15f5 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -79,83 +79,6 @@ struct virtnet_ctl;
*/
#define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
-/* The feature bitmap for virtio net */
-#define VIRTIO_NET_F_CSUM 0 /* Host handles pkts w/ partial csum */
-#define VIRTIO_NET_F_GUEST_CSUM 1 /* Guest handles pkts w/ partial csum */
-#define VIRTIO_NET_F_MTU 3 /* Initial MTU advice. */
-#define VIRTIO_NET_F_MAC 5 /* Host has given MAC address. */
-#define VIRTIO_NET_F_GUEST_TSO4 7 /* Guest can handle TSOv4 in. */
-#define VIRTIO_NET_F_GUEST_TSO6 8 /* Guest can handle TSOv6 in. */
-#define VIRTIO_NET_F_GUEST_ECN 9 /* Guest can handle TSO[6] w/ ECN in. */
-#define VIRTIO_NET_F_GUEST_UFO 10 /* Guest can handle UFO in. */
-#define VIRTIO_NET_F_HOST_TSO4 11 /* Host can handle TSOv4 in. */
-#define VIRTIO_NET_F_HOST_TSO6 12 /* Host can handle TSOv6 in. */
-#define VIRTIO_NET_F_HOST_ECN 13 /* Host can handle TSO[6] w/ ECN in. */
-#define VIRTIO_NET_F_HOST_UFO 14 /* Host can handle UFO in. */
-#define VIRTIO_NET_F_MRG_RXBUF 15 /* Host can merge receive buffers. */
-#define VIRTIO_NET_F_STATUS 16 /* virtio_net_config.status available */
-#define VIRTIO_NET_F_CTRL_VQ 17 /* Control channel available */
-#define VIRTIO_NET_F_CTRL_RX 18 /* Control channel RX mode support */
-#define VIRTIO_NET_F_CTRL_VLAN 19 /* Control channel VLAN filtering */
-#define VIRTIO_NET_F_CTRL_RX_EXTRA 20 /* Extra RX mode control support */
-#define VIRTIO_NET_F_GUEST_ANNOUNCE 21 /* Guest can announce device on the
- * network */
-#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
- * Steering */
-#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
-
-/* Do we get callbacks when the ring is completely used, even if we've
- * suppressed them? */
-#define VIRTIO_F_NOTIFY_ON_EMPTY 24
-
-/* Can the device handle any descriptor layout? */
-#define VIRTIO_F_ANY_LAYOUT 27
-
-/* We support indirect buffer descriptors */
-#define VIRTIO_RING_F_INDIRECT_DESC 28
-
-#define VIRTIO_F_VERSION_1 32
-#define VIRTIO_F_IOMMU_PLATFORM 33
-#define VIRTIO_F_RING_PACKED 34
-
-/*
- * Some VirtIO feature bits (currently bits 28 through 31) are
- * reserved for the transport being used (eg. virtio_ring), the
- * rest are per-device feature bits.
- */
-#define VIRTIO_TRANSPORT_F_START 28
-#define VIRTIO_TRANSPORT_F_END 34
-
-/*
- * Inorder feature indicates that all buffers are used by the device
- * in the same order in which they have been made available.
- */
-#define VIRTIO_F_IN_ORDER 35
-
-/*
- * This feature indicates that memory accesses by the driver and the device
- * are ordered in a way described by the platform.
- */
-#define VIRTIO_F_ORDER_PLATFORM 36
-
-/*
- * This feature indicates that the driver passes extra data (besides
- * identifying the virtqueue) in its device notifications.
- */
-#define VIRTIO_F_NOTIFICATION_DATA 38
-
-/* Device set linkspeed and duplex */
-#define VIRTIO_NET_F_SPEED_DUPLEX 63
-
-/* The Guest publishes the used index for which it expects an interrupt
- * at the end of the avail ring. Host should ignore the avail->flags field. */
-/* The Host publishes the avail index for which it expects a kick
- * at the end of the used ring. Guest should ignore the used->flags field. */
-#define VIRTIO_RING_F_EVENT_IDX 29
-
-#define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
-#define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
-
/*
* Maximum number of virtqueues per device.
*/
@@ -271,17 +194,6 @@ enum virtio_msix_status {
VIRTIO_MSIX_ENABLED = 2
};
-static inline int
-vtpci_with_feature(struct virtio_hw *hw, uint64_t bit)
-{
- return (hw->guest_features & (1ULL << bit)) != 0;
-}
-
-static inline int
-vtpci_packed_queue(struct virtio_hw *hw)
-{
- return vtpci_with_feature(hw, VIRTIO_F_RING_PACKED);
-}
/*
* Function declaration from virtio_pci.c
@@ -294,8 +206,6 @@ void vtpci_reinit_complete(struct virtio_hw *);
uint8_t vtpci_get_status(struct virtio_hw *);
void vtpci_set_status(struct virtio_hw *, uint8_t);
-uint64_t vtpci_negotiate_features(struct virtio_hw *, uint64_t);
-
void vtpci_write_dev_config(struct virtio_hw *, size_t, const void *, int);
void vtpci_read_dev_config(struct virtio_hw *, size_t, void *, int);
diff --git a/drivers/net/virtio/virtio_ring.h b/drivers/net/virtio/virtio_ring.h
index 0f6574f684..17a56b0a73 100644
--- a/drivers/net/virtio/virtio_ring.h
+++ b/drivers/net/virtio/virtio_ring.h
@@ -133,7 +133,7 @@ vring_size(struct virtio_hw *hw, unsigned int num, unsigned long align)
{
size_t size;
- if (vtpci_packed_queue(hw)) {
+ if (virtio_with_packed_queue(hw)) {
size = num * sizeof(struct vring_packed_desc);
size += sizeof(struct vring_packed_desc_event);
size = RTE_ALIGN_CEIL(size, align);
diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index 93fe856cbd..10989118b0 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -685,14 +685,14 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
struct rte_mbuf *m;
uint16_t desc_idx;
int error, nbufs, i;
- bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
+ bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
PMD_INIT_FUNC_TRACE();
/* Allocate blank mbufs for the each rx descriptor */
nbufs = 0;
- if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
+ if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
for (desc_idx = 0; desc_idx < vq->vq_nentries;
desc_idx++) {
vq->vq_split.ring.avail->ring[desc_idx] = desc_idx;
@@ -710,12 +710,12 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
&rxvq->fake_mbuf;
}
- if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
+ if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
while (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
virtio_rxq_rearm_vec(rxvq);
nbufs += RTE_VIRTIO_VPMD_RX_REARM_THRESH;
}
- } else if (!vtpci_packed_queue(vq->hw) && in_order) {
+ } else if (!virtio_with_packed_queue(vq->hw) && in_order) {
if ((!virtqueue_full(vq))) {
uint16_t free_cnt = vq->vq_free_cnt;
struct rte_mbuf *pkts[free_cnt];
@@ -741,7 +741,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
break;
/* Enqueue allocated buffers */
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
error = virtqueue_enqueue_recv_refill_packed(vq,
&m, 1);
else
@@ -754,7 +754,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
nbufs++;
}
- if (!vtpci_packed_queue(vq->hw))
+ if (!virtio_with_packed_queue(vq->hw))
vq_update_avail_idx(vq);
}
@@ -829,8 +829,8 @@ virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
PMD_INIT_FUNC_TRACE();
- if (!vtpci_packed_queue(hw)) {
- if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER))
+ if (!virtio_with_packed_queue(hw)) {
+ if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER))
vq->vq_split.ring.desc[vq->vq_nentries - 1].next = 0;
}
@@ -847,7 +847,7 @@ virtio_discard_rxbuf(struct virtqueue *vq, struct rte_mbuf *m)
* Requeue the discarded mbuf. This should always be
* successful since it was just dequeued.
*/
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
error = virtqueue_enqueue_recv_refill_packed(vq, &m, 1);
else
error = virtqueue_enqueue_recv_refill(vq, &m, 1);
@@ -1209,7 +1209,7 @@ virtio_recv_pkts_inorder(void *rx_queue,
((char *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM
- hdr_size);
- if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
+ if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
seg_num = header->num_buffers;
if (seg_num == 0)
seg_num = 1;
@@ -1735,7 +1735,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf **tx_pkts,
struct virtio_hw *hw = vq->hw;
uint16_t hdr_size = hw->vtnet_hdr_size;
uint16_t nb_tx = 0;
- bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
+ bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
if (unlikely(hw->started == 0 && tx_pkts != hw->inject_pkts))
return nb_tx;
@@ -1754,8 +1754,8 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf **tx_pkts,
int can_push = 0, use_indirect = 0, slots, need;
/* optimize ring usage */
- if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
- vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
+ if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
+ virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
rte_mbuf_refcnt_read(txm) == 1 &&
RTE_MBUF_DIRECT(txm) &&
txm->nb_segs == 1 &&
@@ -1763,7 +1763,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf **tx_pkts,
rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
__alignof__(struct virtio_net_hdr_mrg_rxbuf)))
can_push = 1;
- else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
+ else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
use_indirect = 1;
/* How many main ring entries are needed to this Tx?
@@ -1835,8 +1835,8 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
int can_push = 0, use_indirect = 0, slots, need;
/* optimize ring usage */
- if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
- vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
+ if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
+ virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
rte_mbuf_refcnt_read(txm) == 1 &&
RTE_MBUF_DIRECT(txm) &&
txm->nb_segs == 1 &&
@@ -1844,7 +1844,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
__alignof__(struct virtio_net_hdr_mrg_rxbuf)))
can_push = 1;
- else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
+ else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
use_indirect = 1;
@@ -1937,8 +1937,8 @@ virtio_xmit_pkts_inorder(void *tx_queue,
int slots;
/* optimize ring usage */
- if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
- vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
+ if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
+ virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
rte_mbuf_refcnt_read(txm) == 1 &&
RTE_MBUF_DIRECT(txm) &&
txm->nb_segs == 1 &&
diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c b/drivers/net/virtio/virtio_rxtx_packed_avx.c
index a6a49ec439..c272766a9f 100644
--- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
+++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
@@ -211,14 +211,14 @@ virtqueue_enqueue_single_packed_vec(struct virtnet_tx *txvq,
int16_t need;
/* optimize ring usage */
- if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
- vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
+ if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
+ virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
rte_mbuf_refcnt_read(txm) == 1 &&
RTE_MBUF_DIRECT(txm) &&
txm->nb_segs == 1 &&
rte_pktmbuf_headroom(txm) >= hdr_size)
can_push = 1;
- else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
+ else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
use_indirect = 1;
/* How many main ring entries are needed to this Tx?
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index 79b8446f8e..eade702c5c 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -16,7 +16,7 @@
#include "vhost_kernel_tap.h"
#include "../virtio_logs.h"
-#include "../virtio_pci.h"
+#include "../virtio.h"
int
vhost_kernel_tap_set_offload(int fd, uint64_t features)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 14468ddf52..d05613ba3b 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -122,7 +122,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
dev->features &= dev->device_features;
/* For packed ring, resetting queues is required in reconnection. */
- if (vtpci_packed_queue(hw) &&
+ if (virtio_with_packed_queue(hw) &&
(old_status & VIRTIO_CONFIG_STATUS_DRIVER_OK)) {
PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped"
" when packed ring reconnecting.");
@@ -423,7 +423,7 @@ virtio_user_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- if (vtpci_packed_queue(hw))
+ if (virtio_with_packed_queue(hw))
virtio_user_setup_queue_packed(vq, dev);
else
virtio_user_setup_queue_split(vq, dev);
@@ -456,7 +456,7 @@ virtio_user_notify_queue(struct virtio_hw *hw, struct virtqueue *vq)
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
if (hw->cvq && (hw->cvq->vq == vq)) {
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
virtio_user_handle_cq_packed(dev, vq->vq_queue_index);
else
virtio_user_handle_cq(dev, vq->vq_queue_index);
diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
index 2702e120ee..59a2cb6599 100644
--- a/drivers/net/virtio/virtqueue.c
+++ b/drivers/net/virtio/virtqueue.c
@@ -32,7 +32,7 @@ virtqueue_detach_unused(struct virtqueue *vq)
end = (vq->vq_avail_idx + vq->vq_free_cnt) & (vq->vq_nentries - 1);
for (idx = 0; idx < vq->vq_nentries; idx++) {
- if (hw->use_vec_rx && !vtpci_packed_queue(hw) &&
+ if (hw->use_vec_rx && !virtio_with_packed_queue(hw) &&
type == VTNET_RQ) {
if (start <= end && idx >= start && idx < end)
continue;
@@ -137,7 +137,7 @@ virtqueue_rxvq_flush(struct virtqueue *vq)
{
struct virtio_hw *hw = vq->hw;
- if (vtpci_packed_queue(hw))
+ if (virtio_with_packed_queue(hw))
virtqueue_rxvq_flush_packed(vq);
else
virtqueue_rxvq_flush_split(vq);
diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
index 9d2089766b..6c1df6f8e5 100644
--- a/drivers/net/virtio/virtqueue.h
+++ b/drivers/net/virtio/virtqueue.h
@@ -12,7 +12,7 @@
#include <rte_mempool.h>
#include <rte_net.h>
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtio_ring.h"
#include "virtio_logs.h"
#include "virtio_rxtx.h"
@@ -386,7 +386,7 @@ virtqueue_disable_intr_split(struct virtqueue *vq)
static inline void
virtqueue_disable_intr(struct virtqueue *vq)
{
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
virtqueue_disable_intr_packed(vq);
else
virtqueue_disable_intr_split(vq);
@@ -420,7 +420,7 @@ virtqueue_enable_intr_split(struct virtqueue *vq)
static inline void
virtqueue_enable_intr(struct virtqueue *vq)
{
- if (vtpci_packed_queue(vq->hw))
+ if (virtio_with_packed_queue(vq->hw))
virtqueue_enable_intr_packed(vq);
else
virtqueue_enable_intr_split(vq);
@@ -573,7 +573,7 @@ virtqueue_notify(struct virtqueue *vq)
used_idx = __atomic_load_n(&(vq)->vq_split.ring.used->idx, \
__ATOMIC_RELAXED); \
nused = (uint16_t)(used_idx - (vq)->vq_used_cons_idx); \
- if (vtpci_packed_queue((vq)->hw)) { \
+ if (virtio_with_packed_queue((vq)->hw)) { \
PMD_INIT_LOG(DEBUG, \
"VQ: - size=%d; free=%d; used_cons_idx=%d; avail_idx=%d;" \
" cached_flags=0x%x; used_wrap_counter=%d", \
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header Maxime Coquelin
@ 2020-12-30 3:14 ` Xia, Chenbo
2021-01-14 8:40 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:14 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 17/40] net/virtio: move features definition to generic header
>
> This patch moves all the Virtio definition to the generic
> header. It also renames some helpers to no more reference
> PCI.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/meson.build | 3 +-
> drivers/net/virtio/virtio.c | 22 ++++
> drivers/net/virtio/virtio.h | 94 +++++++++++++++
> drivers/net/virtio/virtio_ethdev.c | 110 +++++++++---------
> drivers/net/virtio/virtio_pci.c | 21 +---
> drivers/net/virtio/virtio_pci.h | 90 --------------
> drivers/net/virtio/virtio_ring.h | 2 +-
> drivers/net/virtio/virtio_rxtx.c | 38 +++---
> drivers/net/virtio/virtio_rxtx_packed_avx.c | 6 +-
> .../net/virtio/virtio_user/vhost_kernel_tap.c | 2 +-
> drivers/net/virtio/virtio_user_ethdev.c | 6 +-
> drivers/net/virtio/virtqueue.c | 4 +-
> drivers/net/virtio/virtqueue.h | 8 +-
> 13 files changed, 209 insertions(+), 197 deletions(-)
> create mode 100644 drivers/net/virtio/virtio.c
>
> diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
> index 0b62418f33..7de41cd04d 100644
> --- a/drivers/net/virtio/meson.build
> +++ b/drivers/net/virtio/meson.build
> @@ -1,7 +1,8 @@
> # SPDX-License-Identifier: BSD-3-Clause
> # Copyright(c) 2018 Intel Corporation
>
> -sources += files('virtio_ethdev.c',
> +sources += files('virtio.c',
> + 'virtio_ethdev.c',
Better align the file names 😊
> 'virtio_pci_ethdev.c',
> 'virtio_pci.c',
> 'virtio_rxtx.c',
> diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
> new file mode 100644
> index 0000000000..d8d6bf7add
> --- /dev/null
> +++ b/drivers/net/virtio/virtio.c
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2020 Red Hat, Inc.
> + */
> +
> +#include "virtio.h"
> +
> +uint64_t
> +virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
> +{
> + uint64_t features;
[snip]
> @@ -1664,9 +1664,9 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>
> /* Setting up rx_header size for the device */
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
> - vtpci_with_feature(hw, VIRTIO_F_RING_PACKED))
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
> + virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
> + virtio_with_feature(hw, VIRTIO_F_RING_PACKED))
There are mixed usage of virtio_with_packed_queue and virtio_with_features(hw,
VIRTIO_F_RING_PACKED). I think we should use only one. Since virtio_with_packed_queue
is introduced, should we only keep this one? What do you think?
Thanks
Chenbo
> hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
> else
> hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr);
> @@ -1681,7 +1681,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> hw->mac_addr[3], hw->mac_addr[4], hw->mac_addr[5]);
>
> if (hw->speed == ETH_SPEED_NUM_UNKNOWN) {
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
> config = &local_config;
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config, speed),
> @@ -1697,14 +1697,14 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> hw->duplex = ETH_LINK_FULL_DUPLEX;
> PMD_INIT_LOG(DEBUG, "link speed = %d, duplex = %d",
> hw->speed, hw->duplex);
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
> config = &local_config;
>
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config, mac),
> &config->mac, sizeof(config->mac));
>
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config, status),
> &config->status, sizeof(config->status));
> @@ -1714,7 +1714,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> config->status = 0;
> }
>
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MQ)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MQ)) {
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config,
> max_virtqueue_pairs),
> &config->max_virtqueue_pairs,
> @@ -1727,7 +1727,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
>
> hw->max_queue_pairs = config->max_virtqueue_pairs;
>
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MTU)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MTU)) {
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config, mtu),
> &config->mtu,
> @@ -1838,7 +1838,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
> goto err_virtio_init;
>
> if (vectorized) {
> - if (!vtpci_packed_queue(hw)) {
> + if (!virtio_with_packed_queue(hw)) {
> hw->use_vec_rx = 1;
> } else {
> #if !defined(CC_AVX512_SUPPORT)
> @@ -1965,17 +1965,17 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs,
> uint32_t *speed, int *vect
> static bool
> rx_offload_enabled(struct virtio_hw *hw)
> {
> - return vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
> - vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
> - vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
> + return virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
> + virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
> + virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
> }
>
> static bool
> tx_offload_enabled(struct virtio_hw *hw)
> {
> - return vtpci_with_feature(hw, VIRTIO_NET_F_CSUM) ||
> - vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
> - vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
> + return virtio_with_feature(hw, VIRTIO_NET_F_CSUM) ||
> + virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
> + virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
> }
>
> /*
> @@ -2048,29 +2048,29 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>
> if ((rx_offloads & (DEV_RX_OFFLOAD_UDP_CKSUM |
> DEV_RX_OFFLOAD_TCP_CKSUM)) &&
> - !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
> + !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
> PMD_DRV_LOG(ERR,
> "rx checksum not available on this host");
> return -ENOTSUP;
> }
>
> if ((rx_offloads & DEV_RX_OFFLOAD_TCP_LRO) &&
> - (!vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
> - !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
> + (!virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
> + !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
> PMD_DRV_LOG(ERR,
> "Large Receive Offload not available on this host");
> return -ENOTSUP;
> }
>
> /* start control queue */
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
> + if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
> virtio_dev_cq_start(dev);
>
> if (rx_offloads & DEV_RX_OFFLOAD_VLAN_STRIP)
> hw->vlan_strip = 1;
>
> - if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER)
> - && !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
> + if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
> + !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
> PMD_DRV_LOG(ERR,
> "vlan filtering not available on this host");
> return -ENOTSUP;
> @@ -2087,12 +2087,12 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> return -EBUSY;
> }
>
> - if (vtpci_packed_queue(hw)) {
> + if (virtio_with_packed_queue(hw)) {
> #if defined(RTE_ARCH_X86_64) && defined(CC_AVX512_SUPPORT)
> if ((hw->use_vec_rx || hw->use_vec_tx) &&
> (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) ||
> - !vtpci_with_feature(hw, VIRTIO_F_IN_ORDER) ||
> - !vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
> + !virtio_with_feature(hw, VIRTIO_F_IN_ORDER) ||
> + !virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
> rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_512)) {
> PMD_DRV_LOG(INFO,
> "disabled packed ring vectorized path for requirements
> not met");
> @@ -2105,7 +2105,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> #endif
>
> if (hw->use_vec_rx) {
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> PMD_DRV_LOG(INFO,
> "disabled packed ring vectorized rx for
> mrg_rxbuf enabled");
> hw->use_vec_rx = 0;
> @@ -2118,7 +2118,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> }
> }
> } else {
> - if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) {
> + if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER)) {
> hw->use_inorder_tx = 1;
> hw->use_inorder_rx = 1;
> hw->use_vec_rx = 0;
> @@ -2132,7 +2132,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> hw->use_vec_rx = 0;
> }
> #endif
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> PMD_DRV_LOG(INFO,
> "disabled split ring vectorized rx for mrg_rxbuf
> enabled");
> hw->use_vec_rx = 0;
> @@ -2350,7 +2350,7 @@ virtio_dev_link_update(struct rte_eth_dev *dev,
> __rte_unused int wait_to_complet
> if (!hw->started) {
> link.link_status = ETH_LINK_DOWN;
> link.link_speed = ETH_SPEED_NUM_NONE;
> - } else if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> + } else if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> PMD_INIT_LOG(DEBUG, "Get link status from hw");
> vtpci_read_dev_config(hw,
> offsetof(struct virtio_net_config, status),
> @@ -2381,7 +2381,7 @@ virtio_dev_vlan_offload_set(struct rte_eth_dev *dev, int
> mask)
>
> if (mask & ETH_VLAN_FILTER_MASK) {
> if ((offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
> - !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
> + !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
>
> PMD_DRV_LOG(NOTICE,
> "vlan filtering not available on this host");
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index df69fcdd45..9c07ebad00 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -356,7 +356,7 @@ modern_set_features(struct virtio_hw *hw, uint64_t
> features)
> static int
> modern_features_ok(struct virtio_hw *hw)
> {
> - if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
> + if (!virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
> PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
> return -1;
> }
> @@ -479,12 +479,12 @@ modern_notify_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> {
> uint32_t notify_data;
>
> - if (!vtpci_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
> + if (!virtio_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
> rte_write16(vq->vq_queue_index, vq->notify_addr);
> return;
> }
>
> - if (vtpci_with_feature(hw, VIRTIO_F_RING_PACKED)) {
> + if (virtio_with_feature(hw, VIRTIO_F_RING_PACKED)) {
> /*
> * Bit[0:15]: vq queue index
> * Bit[16:30]: avail index
> @@ -548,21 +548,6 @@ vtpci_write_dev_config(struct virtio_hw *hw, size_t
> offset,
> VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
> }
>
> -uint64_t
> -vtpci_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
> -{
> - uint64_t features;
> -
> - /*
> - * Limit negotiated features to what the driver, virtqueue, and
> - * host all support.
> - */
> - features = host_features & hw->guest_features;
> - VIRTIO_OPS(hw)->set_features(hw, features);
> -
> - return features;
> -}
> -
> void
> vtpci_reset(struct virtio_hw *hw)
> {
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 8b07c4a369..b02e5c15f5 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -79,83 +79,6 @@ struct virtnet_ctl;
> */
> #define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
>
> -/* The feature bitmap for virtio net */
> -#define VIRTIO_NET_F_CSUM 0 /* Host handles pkts w/ partial csum */
> -#define VIRTIO_NET_F_GUEST_CSUM 1 /* Guest handles pkts w/ partial
> csum */
> -#define VIRTIO_NET_F_MTU 3 /* Initial MTU advice. */
> -#define VIRTIO_NET_F_MAC 5 /* Host has given MAC address. */
> -#define VIRTIO_NET_F_GUEST_TSO4 7 /* Guest can handle TSOv4 in. */
> -#define VIRTIO_NET_F_GUEST_TSO6 8 /* Guest can handle TSOv6 in. */
> -#define VIRTIO_NET_F_GUEST_ECN 9 /* Guest can handle TSO[6] w/ ECN in.
> */
> -#define VIRTIO_NET_F_GUEST_UFO 10 /* Guest can handle UFO in. */
> -#define VIRTIO_NET_F_HOST_TSO4 11 /* Host can handle TSOv4 in. */
> -#define VIRTIO_NET_F_HOST_TSO6 12 /* Host can handle TSOv6 in. */
> -#define VIRTIO_NET_F_HOST_ECN 13 /* Host can handle TSO[6] w/ ECN in.
> */
> -#define VIRTIO_NET_F_HOST_UFO 14 /* Host can handle UFO in. */
> -#define VIRTIO_NET_F_MRG_RXBUF 15 /* Host can merge receive buffers.
> */
> -#define VIRTIO_NET_F_STATUS 16 /* virtio_net_config.status available */
> -#define VIRTIO_NET_F_CTRL_VQ 17 /* Control channel available */
> -#define VIRTIO_NET_F_CTRL_RX 18 /* Control channel RX mode support */
> -#define VIRTIO_NET_F_CTRL_VLAN 19 /* Control channel VLAN filtering */
> -#define VIRTIO_NET_F_CTRL_RX_EXTRA 20 /* Extra RX mode control support */
> -#define VIRTIO_NET_F_GUEST_ANNOUNCE 21 /* Guest can announce device on the
> - * network */
> -#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
> - * Steering */
> -#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
> -
> -/* Do we get callbacks when the ring is completely used, even if we've
> - * suppressed them? */
> -#define VIRTIO_F_NOTIFY_ON_EMPTY 24
> -
> -/* Can the device handle any descriptor layout? */
> -#define VIRTIO_F_ANY_LAYOUT 27
> -
> -/* We support indirect buffer descriptors */
> -#define VIRTIO_RING_F_INDIRECT_DESC 28
> -
> -#define VIRTIO_F_VERSION_1 32
> -#define VIRTIO_F_IOMMU_PLATFORM 33
> -#define VIRTIO_F_RING_PACKED 34
> -
> -/*
> - * Some VirtIO feature bits (currently bits 28 through 31) are
> - * reserved for the transport being used (eg. virtio_ring), the
> - * rest are per-device feature bits.
> - */
> -#define VIRTIO_TRANSPORT_F_START 28
> -#define VIRTIO_TRANSPORT_F_END 34
> -
> -/*
> - * Inorder feature indicates that all buffers are used by the device
> - * in the same order in which they have been made available.
> - */
> -#define VIRTIO_F_IN_ORDER 35
> -
> -/*
> - * This feature indicates that memory accesses by the driver and the device
> - * are ordered in a way described by the platform.
> - */
> -#define VIRTIO_F_ORDER_PLATFORM 36
> -
> -/*
> - * This feature indicates that the driver passes extra data (besides
> - * identifying the virtqueue) in its device notifications.
> - */
> -#define VIRTIO_F_NOTIFICATION_DATA 38
> -
> -/* Device set linkspeed and duplex */
> -#define VIRTIO_NET_F_SPEED_DUPLEX 63
> -
> -/* The Guest publishes the used index for which it expects an interrupt
> - * at the end of the avail ring. Host should ignore the avail->flags field.
> */
> -/* The Host publishes the avail index for which it expects a kick
> - * at the end of the used ring. Guest should ignore the used->flags field. */
> -#define VIRTIO_RING_F_EVENT_IDX 29
> -
> -#define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
> -#define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
> -
> /*
> * Maximum number of virtqueues per device.
> */
> @@ -271,17 +194,6 @@ enum virtio_msix_status {
> VIRTIO_MSIX_ENABLED = 2
> };
>
> -static inline int
> -vtpci_with_feature(struct virtio_hw *hw, uint64_t bit)
> -{
> - return (hw->guest_features & (1ULL << bit)) != 0;
> -}
> -
> -static inline int
> -vtpci_packed_queue(struct virtio_hw *hw)
> -{
> - return vtpci_with_feature(hw, VIRTIO_F_RING_PACKED);
> -}
>
> /*
> * Function declaration from virtio_pci.c
> @@ -294,8 +206,6 @@ void vtpci_reinit_complete(struct virtio_hw *);
> uint8_t vtpci_get_status(struct virtio_hw *);
> void vtpci_set_status(struct virtio_hw *, uint8_t);
>
> -uint64_t vtpci_negotiate_features(struct virtio_hw *, uint64_t);
> -
> void vtpci_write_dev_config(struct virtio_hw *, size_t, const void *, int);
>
> void vtpci_read_dev_config(struct virtio_hw *, size_t, void *, int);
> diff --git a/drivers/net/virtio/virtio_ring.h
> b/drivers/net/virtio/virtio_ring.h
> index 0f6574f684..17a56b0a73 100644
> --- a/drivers/net/virtio/virtio_ring.h
> +++ b/drivers/net/virtio/virtio_ring.h
> @@ -133,7 +133,7 @@ vring_size(struct virtio_hw *hw, unsigned int num,
> unsigned long align)
> {
> size_t size;
>
> - if (vtpci_packed_queue(hw)) {
> + if (virtio_with_packed_queue(hw)) {
> size = num * sizeof(struct vring_packed_desc);
> size += sizeof(struct vring_packed_desc_event);
> size = RTE_ALIGN_CEIL(size, align);
> diff --git a/drivers/net/virtio/virtio_rxtx.c
> b/drivers/net/virtio/virtio_rxtx.c
> index 93fe856cbd..10989118b0 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -685,14 +685,14 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
> uint16_t queue_idx)
> struct rte_mbuf *m;
> uint16_t desc_idx;
> int error, nbufs, i;
> - bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
> + bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
>
> PMD_INIT_FUNC_TRACE();
>
> /* Allocate blank mbufs for the each rx descriptor */
> nbufs = 0;
>
> - if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
> for (desc_idx = 0; desc_idx < vq->vq_nentries;
> desc_idx++) {
> vq->vq_split.ring.avail->ring[desc_idx] = desc_idx;
> @@ -710,12 +710,12 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
> uint16_t queue_idx)
> &rxvq->fake_mbuf;
> }
>
> - if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
> while (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
> virtio_rxq_rearm_vec(rxvq);
> nbufs += RTE_VIRTIO_VPMD_RX_REARM_THRESH;
> }
> - } else if (!vtpci_packed_queue(vq->hw) && in_order) {
> + } else if (!virtio_with_packed_queue(vq->hw) && in_order) {
> if ((!virtqueue_full(vq))) {
> uint16_t free_cnt = vq->vq_free_cnt;
> struct rte_mbuf *pkts[free_cnt];
> @@ -741,7 +741,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
> uint16_t queue_idx)
> break;
>
> /* Enqueue allocated buffers */
> - if (vtpci_packed_queue(vq->hw))
> + if (virtio_with_packed_queue(vq->hw))
> error = virtqueue_enqueue_recv_refill_packed(vq,
> &m, 1);
> else
> @@ -754,7 +754,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
> uint16_t queue_idx)
> nbufs++;
> }
>
> - if (!vtpci_packed_queue(vq->hw))
> + if (!virtio_with_packed_queue(vq->hw))
> vq_update_avail_idx(vq);
> }
>
> @@ -829,8 +829,8 @@ virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
>
> PMD_INIT_FUNC_TRACE();
>
> - if (!vtpci_packed_queue(hw)) {
> - if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER))
> + if (!virtio_with_packed_queue(hw)) {
> + if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER))
> vq->vq_split.ring.desc[vq->vq_nentries - 1].next = 0;
> }
>
> @@ -847,7 +847,7 @@ virtio_discard_rxbuf(struct virtqueue *vq, struct rte_mbuf
> *m)
> * Requeue the discarded mbuf. This should always be
> * successful since it was just dequeued.
> */
> - if (vtpci_packed_queue(vq->hw))
> + if (virtio_with_packed_queue(vq->hw))
> error = virtqueue_enqueue_recv_refill_packed(vq, &m, 1);
> else
> error = virtqueue_enqueue_recv_refill(vq, &m, 1);
> @@ -1209,7 +1209,7 @@ virtio_recv_pkts_inorder(void *rx_queue,
> ((char *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM
> - hdr_size);
>
> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
> seg_num = header->num_buffers;
> if (seg_num == 0)
> seg_num = 1;
> @@ -1735,7 +1735,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> struct virtio_hw *hw = vq->hw;
> uint16_t hdr_size = hw->vtnet_hdr_size;
> uint16_t nb_tx = 0;
> - bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
> + bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
>
> if (unlikely(hw->started == 0 && tx_pkts != hw->inject_pkts))
> return nb_tx;
> @@ -1754,8 +1754,8 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> int can_push = 0, use_indirect = 0, slots, need;
>
> /* optimize ring usage */
> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> rte_mbuf_refcnt_read(txm) == 1 &&
> RTE_MBUF_DIRECT(txm) &&
> txm->nb_segs == 1 &&
> @@ -1763,7 +1763,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
> __alignof__(struct virtio_net_hdr_mrg_rxbuf)))
> can_push = 1;
> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
> use_indirect = 1;
> /* How many main ring entries are needed to this Tx?
> @@ -1835,8 +1835,8 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> **tx_pkts, uint16_t nb_pkts)
> int can_push = 0, use_indirect = 0, slots, need;
>
> /* optimize ring usage */
> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> rte_mbuf_refcnt_read(txm) == 1 &&
> RTE_MBUF_DIRECT(txm) &&
> txm->nb_segs == 1 &&
> @@ -1844,7 +1844,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> **tx_pkts, uint16_t nb_pkts)
> rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
> __alignof__(struct virtio_net_hdr_mrg_rxbuf)))
> can_push = 1;
> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
> use_indirect = 1;
>
> @@ -1937,8 +1937,8 @@ virtio_xmit_pkts_inorder(void *tx_queue,
> int slots;
>
> /* optimize ring usage */
> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> rte_mbuf_refcnt_read(txm) == 1 &&
> RTE_MBUF_DIRECT(txm) &&
> txm->nb_segs == 1 &&
> diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> index a6a49ec439..c272766a9f 100644
> --- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> @@ -211,14 +211,14 @@ virtqueue_enqueue_single_packed_vec(struct virtnet_tx
> *txvq,
> int16_t need;
>
> /* optimize ring usage */
> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
> rte_mbuf_refcnt_read(txm) == 1 &&
> RTE_MBUF_DIRECT(txm) &&
> txm->nb_segs == 1 &&
> rte_pktmbuf_headroom(txm) >= hdr_size)
> can_push = 1;
> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
> use_indirect = 1;
> /* How many main ring entries are needed to this Tx?
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> index 79b8446f8e..eade702c5c 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> @@ -16,7 +16,7 @@
>
> #include "vhost_kernel_tap.h"
> #include "../virtio_logs.h"
> -#include "../virtio_pci.h"
> +#include "../virtio.h"
>
> int
> vhost_kernel_tap_set_offload(int fd, uint64_t features)
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 14468ddf52..d05613ba3b 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -122,7 +122,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
> dev->features &= dev->device_features;
>
> /* For packed ring, resetting queues is required in reconnection. */
> - if (vtpci_packed_queue(hw) &&
> + if (virtio_with_packed_queue(hw) &&
> (old_status & VIRTIO_CONFIG_STATUS_DRIVER_OK)) {
> PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped"
> " when packed ring reconnecting.");
> @@ -423,7 +423,7 @@ virtio_user_setup_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> {
> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>
> - if (vtpci_packed_queue(hw))
> + if (virtio_with_packed_queue(hw))
> virtio_user_setup_queue_packed(vq, dev);
> else
> virtio_user_setup_queue_split(vq, dev);
> @@ -456,7 +456,7 @@ virtio_user_notify_queue(struct virtio_hw *hw, struct
> virtqueue *vq)
> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>
> if (hw->cvq && (hw->cvq->vq == vq)) {
> - if (vtpci_packed_queue(vq->hw))
> + if (virtio_with_packed_queue(vq->hw))
> virtio_user_handle_cq_packed(dev, vq->vq_queue_index);
> else
> virtio_user_handle_cq(dev, vq->vq_queue_index);
> diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
> index 2702e120ee..59a2cb6599 100644
> --- a/drivers/net/virtio/virtqueue.c
> +++ b/drivers/net/virtio/virtqueue.c
> @@ -32,7 +32,7 @@ virtqueue_detach_unused(struct virtqueue *vq)
> end = (vq->vq_avail_idx + vq->vq_free_cnt) & (vq->vq_nentries - 1);
>
> for (idx = 0; idx < vq->vq_nentries; idx++) {
> - if (hw->use_vec_rx && !vtpci_packed_queue(hw) &&
> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw) &&
> type == VTNET_RQ) {
> if (start <= end && idx >= start && idx < end)
> continue;
> @@ -137,7 +137,7 @@ virtqueue_rxvq_flush(struct virtqueue *vq)
> {
> struct virtio_hw *hw = vq->hw;
>
> - if (vtpci_packed_queue(hw))
> + if (virtio_with_packed_queue(hw))
> virtqueue_rxvq_flush_packed(vq);
> else
> virtqueue_rxvq_flush_split(vq);
> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
> index 9d2089766b..6c1df6f8e5 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -12,7 +12,7 @@
> #include <rte_mempool.h>
> #include <rte_net.h>
>
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtio_ring.h"
> #include "virtio_logs.h"
> #include "virtio_rxtx.h"
> @@ -386,7 +386,7 @@ virtqueue_disable_intr_split(struct virtqueue *vq)
> static inline void
> virtqueue_disable_intr(struct virtqueue *vq)
> {
> - if (vtpci_packed_queue(vq->hw))
> + if (virtio_with_packed_queue(vq->hw))
> virtqueue_disable_intr_packed(vq);
> else
> virtqueue_disable_intr_split(vq);
> @@ -420,7 +420,7 @@ virtqueue_enable_intr_split(struct virtqueue *vq)
> static inline void
> virtqueue_enable_intr(struct virtqueue *vq)
> {
> - if (vtpci_packed_queue(vq->hw))
> + if (virtio_with_packed_queue(vq->hw))
> virtqueue_enable_intr_packed(vq);
> else
> virtqueue_enable_intr_split(vq);
> @@ -573,7 +573,7 @@ virtqueue_notify(struct virtqueue *vq)
> used_idx = __atomic_load_n(&(vq)->vq_split.ring.used->idx, \
> __ATOMIC_RELAXED); \
> nused = (uint16_t)(used_idx - (vq)->vq_used_cons_idx); \
> - if (vtpci_packed_queue((vq)->hw)) { \
> + if (virtio_with_packed_queue((vq)->hw)) { \
> PMD_INIT_LOG(DEBUG, \
> "VQ: - size=%d; free=%d; used_cons_idx=%d; avail_idx=%d;" \
> " cached_flags=0x%x; used_wrap_counter=%d", \
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header
2020-12-30 3:14 ` Xia, Chenbo
@ 2021-01-14 8:40 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 8:40 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 12/30/20 4:14 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 17/40] net/virtio: move features definition to generic header
>>
>> This patch moves all the Virtio definition to the generic
>> header. It also renames some helpers to no more reference
>> PCI.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/meson.build | 3 +-
>> drivers/net/virtio/virtio.c | 22 ++++
>> drivers/net/virtio/virtio.h | 94 +++++++++++++++
>> drivers/net/virtio/virtio_ethdev.c | 110 +++++++++---------
>> drivers/net/virtio/virtio_pci.c | 21 +---
>> drivers/net/virtio/virtio_pci.h | 90 --------------
>> drivers/net/virtio/virtio_ring.h | 2 +-
>> drivers/net/virtio/virtio_rxtx.c | 38 +++---
>> drivers/net/virtio/virtio_rxtx_packed_avx.c | 6 +-
>> .../net/virtio/virtio_user/vhost_kernel_tap.c | 2 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 6 +-
>> drivers/net/virtio/virtqueue.c | 4 +-
>> drivers/net/virtio/virtqueue.h | 8 +-
>> 13 files changed, 209 insertions(+), 197 deletions(-)
>> create mode 100644 drivers/net/virtio/virtio.c
>>
>> diff --git a/drivers/net/virtio/meson.build b/drivers/net/virtio/meson.build
>> index 0b62418f33..7de41cd04d 100644
>> --- a/drivers/net/virtio/meson.build
>> +++ b/drivers/net/virtio/meson.build
>> @@ -1,7 +1,8 @@
>> # SPDX-License-Identifier: BSD-3-Clause
>> # Copyright(c) 2018 Intel Corporation
>>
>> -sources += files('virtio_ethdev.c',
>> +sources += files('virtio.c',
>> + 'virtio_ethdev.c',
>
> Better align the file names 😊
Done.
>
>> 'virtio_pci_ethdev.c',
>> 'virtio_pci.c',
>> 'virtio_rxtx.c',
>> diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
>> new file mode 100644
>> index 0000000000..d8d6bf7add
>> --- /dev/null
>> +++ b/drivers/net/virtio/virtio.c
>> @@ -0,0 +1,22 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2010-2014 Intel Corporation
>> + * Copyright(c) 2020 Red Hat, Inc.
>> + */
>> +
>> +#include "virtio.h"
>> +
>> +uint64_t
>> +virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
>> +{
>> + uint64_t features;
>
> [snip]
>
>> @@ -1664,9 +1664,9 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
>> req_features)
>> eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>>
>> /* Setting up rx_header size for the device */
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
>> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
>> - vtpci_with_feature(hw, VIRTIO_F_RING_PACKED))
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF) ||
>> + virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
>> + virtio_with_feature(hw, VIRTIO_F_RING_PACKED))
>
> There are mixed usage of virtio_with_packed_queue and virtio_with_features(hw,
> VIRTIO_F_RING_PACKED). I think we should use only one. Since virtio_with_packed_queue
> is introduced, should we only keep this one? What do you think?
Yes, it may be better to use one as it make it easier to grep for it.
I will apply your suggestion in v2.
Thanks!
Maxime
> Thanks
> Chenbo
>
>> hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
>> else
>> hw->vtnet_hdr_size = sizeof(struct virtio_net_hdr);
>> @@ -1681,7 +1681,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
>> req_features)
>> hw->mac_addr[3], hw->mac_addr[4], hw->mac_addr[5]);
>>
>> if (hw->speed == ETH_SPEED_NUM_UNKNOWN) {
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
>> config = &local_config;
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config, speed),
>> @@ -1697,14 +1697,14 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
>> uint64_t req_features)
>> hw->duplex = ETH_LINK_FULL_DUPLEX;
>> PMD_INIT_LOG(DEBUG, "link speed = %d, duplex = %d",
>> hw->speed, hw->duplex);
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
>> config = &local_config;
>>
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config, mac),
>> &config->mac, sizeof(config->mac));
>>
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config, status),
>> &config->status, sizeof(config->status));
>> @@ -1714,7 +1714,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
>> req_features)
>> config->status = 0;
>> }
>>
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MQ)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MQ)) {
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config,
>> max_virtqueue_pairs),
>> &config->max_virtqueue_pairs,
>> @@ -1727,7 +1727,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
>> req_features)
>>
>> hw->max_queue_pairs = config->max_virtqueue_pairs;
>>
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MTU)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MTU)) {
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config, mtu),
>> &config->mtu,
>> @@ -1838,7 +1838,7 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>> goto err_virtio_init;
>>
>> if (vectorized) {
>> - if (!vtpci_packed_queue(hw)) {
>> + if (!virtio_with_packed_queue(hw)) {
>> hw->use_vec_rx = 1;
>> } else {
>> #if !defined(CC_AVX512_SUPPORT)
>> @@ -1965,17 +1965,17 @@ virtio_dev_devargs_parse(struct rte_devargs *devargs,
>> uint32_t *speed, int *vect
>> static bool
>> rx_offload_enabled(struct virtio_hw *hw)
>> {
>> - return vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
>> - vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
>> - vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
>> + return virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM) ||
>> + virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
>> + virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6);
>> }
>>
>> static bool
>> tx_offload_enabled(struct virtio_hw *hw)
>> {
>> - return vtpci_with_feature(hw, VIRTIO_NET_F_CSUM) ||
>> - vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
>> - vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
>> + return virtio_with_feature(hw, VIRTIO_NET_F_CSUM) ||
>> + virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO4) ||
>> + virtio_with_feature(hw, VIRTIO_NET_F_HOST_TSO6);
>> }
>>
>> /*
>> @@ -2048,29 +2048,29 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>>
>> if ((rx_offloads & (DEV_RX_OFFLOAD_UDP_CKSUM |
>> DEV_RX_OFFLOAD_TCP_CKSUM)) &&
>> - !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
>> + !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_CSUM)) {
>> PMD_DRV_LOG(ERR,
>> "rx checksum not available on this host");
>> return -ENOTSUP;
>> }
>>
>> if ((rx_offloads & DEV_RX_OFFLOAD_TCP_LRO) &&
>> - (!vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
>> - !vtpci_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
>> + (!virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO4) ||
>> + !virtio_with_feature(hw, VIRTIO_NET_F_GUEST_TSO6))) {
>> PMD_DRV_LOG(ERR,
>> "Large Receive Offload not available on this host");
>> return -ENOTSUP;
>> }
>>
>> /* start control queue */
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ))
>> virtio_dev_cq_start(dev);
>>
>> if (rx_offloads & DEV_RX_OFFLOAD_VLAN_STRIP)
>> hw->vlan_strip = 1;
>>
>> - if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER)
>> - && !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
>> + if ((rx_offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
>> + !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
>> PMD_DRV_LOG(ERR,
>> "vlan filtering not available on this host");
>> return -ENOTSUP;
>> @@ -2087,12 +2087,12 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>> return -EBUSY;
>> }
>>
>> - if (vtpci_packed_queue(hw)) {
>> + if (virtio_with_packed_queue(hw)) {
>> #if defined(RTE_ARCH_X86_64) && defined(CC_AVX512_SUPPORT)
>> if ((hw->use_vec_rx || hw->use_vec_tx) &&
>> (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) ||
>> - !vtpci_with_feature(hw, VIRTIO_F_IN_ORDER) ||
>> - !vtpci_with_feature(hw, VIRTIO_F_VERSION_1) ||
>> + !virtio_with_feature(hw, VIRTIO_F_IN_ORDER) ||
>> + !virtio_with_feature(hw, VIRTIO_F_VERSION_1) ||
>> rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_512)) {
>> PMD_DRV_LOG(INFO,
>> "disabled packed ring vectorized path for requirements
>> not met");
>> @@ -2105,7 +2105,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>> #endif
>>
>> if (hw->use_vec_rx) {
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> PMD_DRV_LOG(INFO,
>> "disabled packed ring vectorized rx for
>> mrg_rxbuf enabled");
>> hw->use_vec_rx = 0;
>> @@ -2118,7 +2118,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>> }
>> }
>> } else {
>> - if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) {
>> + if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER)) {
>> hw->use_inorder_tx = 1;
>> hw->use_inorder_rx = 1;
>> hw->use_vec_rx = 0;
>> @@ -2132,7 +2132,7 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>> hw->use_vec_rx = 0;
>> }
>> #endif
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> PMD_DRV_LOG(INFO,
>> "disabled split ring vectorized rx for mrg_rxbuf
>> enabled");
>> hw->use_vec_rx = 0;
>> @@ -2350,7 +2350,7 @@ virtio_dev_link_update(struct rte_eth_dev *dev,
>> __rte_unused int wait_to_complet
>> if (!hw->started) {
>> link.link_status = ETH_LINK_DOWN;
>> link.link_speed = ETH_SPEED_NUM_NONE;
>> - } else if (vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) {
>> + } else if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
>> PMD_INIT_LOG(DEBUG, "Get link status from hw");
>> vtpci_read_dev_config(hw,
>> offsetof(struct virtio_net_config, status),
>> @@ -2381,7 +2381,7 @@ virtio_dev_vlan_offload_set(struct rte_eth_dev *dev, int
>> mask)
>>
>> if (mask & ETH_VLAN_FILTER_MASK) {
>> if ((offloads & DEV_RX_OFFLOAD_VLAN_FILTER) &&
>> - !vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
>> + !virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VLAN)) {
>>
>> PMD_DRV_LOG(NOTICE,
>> "vlan filtering not available on this host");
>> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
>> index df69fcdd45..9c07ebad00 100644
>> --- a/drivers/net/virtio/virtio_pci.c
>> +++ b/drivers/net/virtio/virtio_pci.c
>> @@ -356,7 +356,7 @@ modern_set_features(struct virtio_hw *hw, uint64_t
>> features)
>> static int
>> modern_features_ok(struct virtio_hw *hw)
>> {
>> - if (!vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) {
>> + if (!virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
>> PMD_INIT_LOG(ERR, "Version 1+ required with modern devices\n");
>> return -1;
>> }
>> @@ -479,12 +479,12 @@ modern_notify_queue(struct virtio_hw *hw, struct
>> virtqueue *vq)
>> {
>> uint32_t notify_data;
>>
>> - if (!vtpci_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
>> + if (!virtio_with_feature(hw, VIRTIO_F_NOTIFICATION_DATA)) {
>> rte_write16(vq->vq_queue_index, vq->notify_addr);
>> return;
>> }
>>
>> - if (vtpci_with_feature(hw, VIRTIO_F_RING_PACKED)) {
>> + if (virtio_with_feature(hw, VIRTIO_F_RING_PACKED)) {
>> /*
>> * Bit[0:15]: vq queue index
>> * Bit[16:30]: avail index
>> @@ -548,21 +548,6 @@ vtpci_write_dev_config(struct virtio_hw *hw, size_t
>> offset,
>> VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
>> }
>>
>> -uint64_t
>> -vtpci_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
>> -{
>> - uint64_t features;
>> -
>> - /*
>> - * Limit negotiated features to what the driver, virtqueue, and
>> - * host all support.
>> - */
>> - features = host_features & hw->guest_features;
>> - VIRTIO_OPS(hw)->set_features(hw, features);
>> -
>> - return features;
>> -}
>> -
>> void
>> vtpci_reset(struct virtio_hw *hw)
>> {
>> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
>> index 8b07c4a369..b02e5c15f5 100644
>> --- a/drivers/net/virtio/virtio_pci.h
>> +++ b/drivers/net/virtio/virtio_pci.h
>> @@ -79,83 +79,6 @@ struct virtnet_ctl;
>> */
>> #define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
>>
>> -/* The feature bitmap for virtio net */
>> -#define VIRTIO_NET_F_CSUM 0 /* Host handles pkts w/ partial csum */
>> -#define VIRTIO_NET_F_GUEST_CSUM 1 /* Guest handles pkts w/ partial
>> csum */
>> -#define VIRTIO_NET_F_MTU 3 /* Initial MTU advice. */
>> -#define VIRTIO_NET_F_MAC 5 /* Host has given MAC address. */
>> -#define VIRTIO_NET_F_GUEST_TSO4 7 /* Guest can handle TSOv4 in. */
>> -#define VIRTIO_NET_F_GUEST_TSO6 8 /* Guest can handle TSOv6 in. */
>> -#define VIRTIO_NET_F_GUEST_ECN 9 /* Guest can handle TSO[6] w/ ECN in.
>> */
>> -#define VIRTIO_NET_F_GUEST_UFO 10 /* Guest can handle UFO in. */
>> -#define VIRTIO_NET_F_HOST_TSO4 11 /* Host can handle TSOv4 in. */
>> -#define VIRTIO_NET_F_HOST_TSO6 12 /* Host can handle TSOv6 in. */
>> -#define VIRTIO_NET_F_HOST_ECN 13 /* Host can handle TSO[6] w/ ECN in.
>> */
>> -#define VIRTIO_NET_F_HOST_UFO 14 /* Host can handle UFO in. */
>> -#define VIRTIO_NET_F_MRG_RXBUF 15 /* Host can merge receive buffers.
>> */
>> -#define VIRTIO_NET_F_STATUS 16 /* virtio_net_config.status available */
>> -#define VIRTIO_NET_F_CTRL_VQ 17 /* Control channel available */
>> -#define VIRTIO_NET_F_CTRL_RX 18 /* Control channel RX mode support */
>> -#define VIRTIO_NET_F_CTRL_VLAN 19 /* Control channel VLAN filtering */
>> -#define VIRTIO_NET_F_CTRL_RX_EXTRA 20 /* Extra RX mode control support */
>> -#define VIRTIO_NET_F_GUEST_ANNOUNCE 21 /* Guest can announce device on the
>> - * network */
>> -#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
>> - * Steering */
>> -#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
>> -
>> -/* Do we get callbacks when the ring is completely used, even if we've
>> - * suppressed them? */
>> -#define VIRTIO_F_NOTIFY_ON_EMPTY 24
>> -
>> -/* Can the device handle any descriptor layout? */
>> -#define VIRTIO_F_ANY_LAYOUT 27
>> -
>> -/* We support indirect buffer descriptors */
>> -#define VIRTIO_RING_F_INDIRECT_DESC 28
>> -
>> -#define VIRTIO_F_VERSION_1 32
>> -#define VIRTIO_F_IOMMU_PLATFORM 33
>> -#define VIRTIO_F_RING_PACKED 34
>> -
>> -/*
>> - * Some VirtIO feature bits (currently bits 28 through 31) are
>> - * reserved for the transport being used (eg. virtio_ring), the
>> - * rest are per-device feature bits.
>> - */
>> -#define VIRTIO_TRANSPORT_F_START 28
>> -#define VIRTIO_TRANSPORT_F_END 34
>> -
>> -/*
>> - * Inorder feature indicates that all buffers are used by the device
>> - * in the same order in which they have been made available.
>> - */
>> -#define VIRTIO_F_IN_ORDER 35
>> -
>> -/*
>> - * This feature indicates that memory accesses by the driver and the device
>> - * are ordered in a way described by the platform.
>> - */
>> -#define VIRTIO_F_ORDER_PLATFORM 36
>> -
>> -/*
>> - * This feature indicates that the driver passes extra data (besides
>> - * identifying the virtqueue) in its device notifications.
>> - */
>> -#define VIRTIO_F_NOTIFICATION_DATA 38
>> -
>> -/* Device set linkspeed and duplex */
>> -#define VIRTIO_NET_F_SPEED_DUPLEX 63
>> -
>> -/* The Guest publishes the used index for which it expects an interrupt
>> - * at the end of the avail ring. Host should ignore the avail->flags field.
>> */
>> -/* The Host publishes the avail index for which it expects a kick
>> - * at the end of the used ring. Guest should ignore the used->flags field. */
>> -#define VIRTIO_RING_F_EVENT_IDX 29
>> -
>> -#define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
>> -#define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
>> -
>> /*
>> * Maximum number of virtqueues per device.
>> */
>> @@ -271,17 +194,6 @@ enum virtio_msix_status {
>> VIRTIO_MSIX_ENABLED = 2
>> };
>>
>> -static inline int
>> -vtpci_with_feature(struct virtio_hw *hw, uint64_t bit)
>> -{
>> - return (hw->guest_features & (1ULL << bit)) != 0;
>> -}
>> -
>> -static inline int
>> -vtpci_packed_queue(struct virtio_hw *hw)
>> -{
>> - return vtpci_with_feature(hw, VIRTIO_F_RING_PACKED);
>> -}
>>
>> /*
>> * Function declaration from virtio_pci.c
>> @@ -294,8 +206,6 @@ void vtpci_reinit_complete(struct virtio_hw *);
>> uint8_t vtpci_get_status(struct virtio_hw *);
>> void vtpci_set_status(struct virtio_hw *, uint8_t);
>>
>> -uint64_t vtpci_negotiate_features(struct virtio_hw *, uint64_t);
>> -
>> void vtpci_write_dev_config(struct virtio_hw *, size_t, const void *, int);
>>
>> void vtpci_read_dev_config(struct virtio_hw *, size_t, void *, int);
>> diff --git a/drivers/net/virtio/virtio_ring.h
>> b/drivers/net/virtio/virtio_ring.h
>> index 0f6574f684..17a56b0a73 100644
>> --- a/drivers/net/virtio/virtio_ring.h
>> +++ b/drivers/net/virtio/virtio_ring.h
>> @@ -133,7 +133,7 @@ vring_size(struct virtio_hw *hw, unsigned int num,
>> unsigned long align)
>> {
>> size_t size;
>>
>> - if (vtpci_packed_queue(hw)) {
>> + if (virtio_with_packed_queue(hw)) {
>> size = num * sizeof(struct vring_packed_desc);
>> size += sizeof(struct vring_packed_desc_event);
>> size = RTE_ALIGN_CEIL(size, align);
>> diff --git a/drivers/net/virtio/virtio_rxtx.c
>> b/drivers/net/virtio/virtio_rxtx.c
>> index 93fe856cbd..10989118b0 100644
>> --- a/drivers/net/virtio/virtio_rxtx.c
>> +++ b/drivers/net/virtio/virtio_rxtx.c
>> @@ -685,14 +685,14 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
>> uint16_t queue_idx)
>> struct rte_mbuf *m;
>> uint16_t desc_idx;
>> int error, nbufs, i;
>> - bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
>> + bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
>>
>> PMD_INIT_FUNC_TRACE();
>>
>> /* Allocate blank mbufs for the each rx descriptor */
>> nbufs = 0;
>>
>> - if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
>> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
>> for (desc_idx = 0; desc_idx < vq->vq_nentries;
>> desc_idx++) {
>> vq->vq_split.ring.avail->ring[desc_idx] = desc_idx;
>> @@ -710,12 +710,12 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
>> uint16_t queue_idx)
>> &rxvq->fake_mbuf;
>> }
>>
>> - if (hw->use_vec_rx && !vtpci_packed_queue(hw)) {
>> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) {
>> while (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
>> virtio_rxq_rearm_vec(rxvq);
>> nbufs += RTE_VIRTIO_VPMD_RX_REARM_THRESH;
>> }
>> - } else if (!vtpci_packed_queue(vq->hw) && in_order) {
>> + } else if (!virtio_with_packed_queue(vq->hw) && in_order) {
>> if ((!virtqueue_full(vq))) {
>> uint16_t free_cnt = vq->vq_free_cnt;
>> struct rte_mbuf *pkts[free_cnt];
>> @@ -741,7 +741,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
>> uint16_t queue_idx)
>> break;
>>
>> /* Enqueue allocated buffers */
>> - if (vtpci_packed_queue(vq->hw))
>> + if (virtio_with_packed_queue(vq->hw))
>> error = virtqueue_enqueue_recv_refill_packed(vq,
>> &m, 1);
>> else
>> @@ -754,7 +754,7 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev,
>> uint16_t queue_idx)
>> nbufs++;
>> }
>>
>> - if (!vtpci_packed_queue(vq->hw))
>> + if (!virtio_with_packed_queue(vq->hw))
>> vq_update_avail_idx(vq);
>> }
>>
>> @@ -829,8 +829,8 @@ virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
>>
>> PMD_INIT_FUNC_TRACE();
>>
>> - if (!vtpci_packed_queue(hw)) {
>> - if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER))
>> + if (!virtio_with_packed_queue(hw)) {
>> + if (virtio_with_feature(hw, VIRTIO_F_IN_ORDER))
>> vq->vq_split.ring.desc[vq->vq_nentries - 1].next = 0;
>> }
>>
>> @@ -847,7 +847,7 @@ virtio_discard_rxbuf(struct virtqueue *vq, struct rte_mbuf
>> *m)
>> * Requeue the discarded mbuf. This should always be
>> * successful since it was just dequeued.
>> */
>> - if (vtpci_packed_queue(vq->hw))
>> + if (virtio_with_packed_queue(vq->hw))
>> error = virtqueue_enqueue_recv_refill_packed(vq, &m, 1);
>> else
>> error = virtqueue_enqueue_recv_refill(vq, &m, 1);
>> @@ -1209,7 +1209,7 @@ virtio_recv_pkts_inorder(void *rx_queue,
>> ((char *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM
>> - hdr_size);
>>
>> - if (vtpci_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> + if (virtio_with_feature(hw, VIRTIO_NET_F_MRG_RXBUF)) {
>> seg_num = header->num_buffers;
>> if (seg_num == 0)
>> seg_num = 1;
>> @@ -1735,7 +1735,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
>> **tx_pkts,
>> struct virtio_hw *hw = vq->hw;
>> uint16_t hdr_size = hw->vtnet_hdr_size;
>> uint16_t nb_tx = 0;
>> - bool in_order = vtpci_with_feature(hw, VIRTIO_F_IN_ORDER);
>> + bool in_order = virtio_with_feature(hw, VIRTIO_F_IN_ORDER);
>>
>> if (unlikely(hw->started == 0 && tx_pkts != hw->inject_pkts))
>> return nb_tx;
>> @@ -1754,8 +1754,8 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
>> **tx_pkts,
>> int can_push = 0, use_indirect = 0, slots, need;
>>
>> /* optimize ring usage */
>> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> rte_mbuf_refcnt_read(txm) == 1 &&
>> RTE_MBUF_DIRECT(txm) &&
>> txm->nb_segs == 1 &&
>> @@ -1763,7 +1763,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf
>> **tx_pkts,
>> rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
>> __alignof__(struct virtio_net_hdr_mrg_rxbuf)))
>> can_push = 1;
>> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
>> use_indirect = 1;
>> /* How many main ring entries are needed to this Tx?
>> @@ -1835,8 +1835,8 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
>> **tx_pkts, uint16_t nb_pkts)
>> int can_push = 0, use_indirect = 0, slots, need;
>>
>> /* optimize ring usage */
>> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> rte_mbuf_refcnt_read(txm) == 1 &&
>> RTE_MBUF_DIRECT(txm) &&
>> txm->nb_segs == 1 &&
>> @@ -1844,7 +1844,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
>> **tx_pkts, uint16_t nb_pkts)
>> rte_is_aligned(rte_pktmbuf_mtod(txm, char *),
>> __alignof__(struct virtio_net_hdr_mrg_rxbuf)))
>> can_push = 1;
>> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
>> use_indirect = 1;
>>
>> @@ -1937,8 +1937,8 @@ virtio_xmit_pkts_inorder(void *tx_queue,
>> int slots;
>>
>> /* optimize ring usage */
>> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> rte_mbuf_refcnt_read(txm) == 1 &&
>> RTE_MBUF_DIRECT(txm) &&
>> txm->nb_segs == 1 &&
>> diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> b/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> index a6a49ec439..c272766a9f 100644
>> --- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> @@ -211,14 +211,14 @@ virtqueue_enqueue_single_packed_vec(struct virtnet_tx
>> *txvq,
>> int16_t need;
>>
>> /* optimize ring usage */
>> - if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> - vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> + if ((virtio_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> + virtio_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>> rte_mbuf_refcnt_read(txm) == 1 &&
>> RTE_MBUF_DIRECT(txm) &&
>> txm->nb_segs == 1 &&
>> rte_pktmbuf_headroom(txm) >= hdr_size)
>> can_push = 1;
>> - else if (vtpci_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> + else if (virtio_with_feature(hw, VIRTIO_RING_F_INDIRECT_DESC) &&
>> txm->nb_segs < VIRTIO_MAX_TX_INDIRECT)
>> use_indirect = 1;
>> /* How many main ring entries are needed to this Tx?
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
>> index 79b8446f8e..eade702c5c 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
>> @@ -16,7 +16,7 @@
>>
>> #include "vhost_kernel_tap.h"
>> #include "../virtio_logs.h"
>> -#include "../virtio_pci.h"
>> +#include "../virtio.h"
>>
>> int
>> vhost_kernel_tap_set_offload(int fd, uint64_t features)
>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
>> b/drivers/net/virtio/virtio_user_ethdev.c
>> index 14468ddf52..d05613ba3b 100644
>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>> @@ -122,7 +122,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>> dev->features &= dev->device_features;
>>
>> /* For packed ring, resetting queues is required in reconnection. */
>> - if (vtpci_packed_queue(hw) &&
>> + if (virtio_with_packed_queue(hw) &&
>> (old_status & VIRTIO_CONFIG_STATUS_DRIVER_OK)) {
>> PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped"
>> " when packed ring reconnecting.");
>> @@ -423,7 +423,7 @@ virtio_user_setup_queue(struct virtio_hw *hw, struct
>> virtqueue *vq)
>> {
>> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>>
>> - if (vtpci_packed_queue(hw))
>> + if (virtio_with_packed_queue(hw))
>> virtio_user_setup_queue_packed(vq, dev);
>> else
>> virtio_user_setup_queue_split(vq, dev);
>> @@ -456,7 +456,7 @@ virtio_user_notify_queue(struct virtio_hw *hw, struct
>> virtqueue *vq)
>> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>>
>> if (hw->cvq && (hw->cvq->vq == vq)) {
>> - if (vtpci_packed_queue(vq->hw))
>> + if (virtio_with_packed_queue(vq->hw))
>> virtio_user_handle_cq_packed(dev, vq->vq_queue_index);
>> else
>> virtio_user_handle_cq(dev, vq->vq_queue_index);
>> diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
>> index 2702e120ee..59a2cb6599 100644
>> --- a/drivers/net/virtio/virtqueue.c
>> +++ b/drivers/net/virtio/virtqueue.c
>> @@ -32,7 +32,7 @@ virtqueue_detach_unused(struct virtqueue *vq)
>> end = (vq->vq_avail_idx + vq->vq_free_cnt) & (vq->vq_nentries - 1);
>>
>> for (idx = 0; idx < vq->vq_nentries; idx++) {
>> - if (hw->use_vec_rx && !vtpci_packed_queue(hw) &&
>> + if (hw->use_vec_rx && !virtio_with_packed_queue(hw) &&
>> type == VTNET_RQ) {
>> if (start <= end && idx >= start && idx < end)
>> continue;
>> @@ -137,7 +137,7 @@ virtqueue_rxvq_flush(struct virtqueue *vq)
>> {
>> struct virtio_hw *hw = vq->hw;
>>
>> - if (vtpci_packed_queue(hw))
>> + if (virtio_with_packed_queue(hw))
>> virtqueue_rxvq_flush_packed(vq);
>> else
>> virtqueue_rxvq_flush_split(vq);
>> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
>> index 9d2089766b..6c1df6f8e5 100644
>> --- a/drivers/net/virtio/virtqueue.h
>> +++ b/drivers/net/virtio/virtqueue.h
>> @@ -12,7 +12,7 @@
>> #include <rte_mempool.h>
>> #include <rte_net.h>
>>
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> #include "virtio_ring.h"
>> #include "virtio_logs.h"
>> #include "virtio_rxtx.h"
>> @@ -386,7 +386,7 @@ virtqueue_disable_intr_split(struct virtqueue *vq)
>> static inline void
>> virtqueue_disable_intr(struct virtqueue *vq)
>> {
>> - if (vtpci_packed_queue(vq->hw))
>> + if (virtio_with_packed_queue(vq->hw))
>> virtqueue_disable_intr_packed(vq);
>> else
>> virtqueue_disable_intr_split(vq);
>> @@ -420,7 +420,7 @@ virtqueue_enable_intr_split(struct virtqueue *vq)
>> static inline void
>> virtqueue_enable_intr(struct virtqueue *vq)
>> {
>> - if (vtpci_packed_queue(vq->hw))
>> + if (virtio_with_packed_queue(vq->hw))
>> virtqueue_enable_intr_packed(vq);
>> else
>> virtqueue_enable_intr_split(vq);
>> @@ -573,7 +573,7 @@ virtqueue_notify(struct virtqueue *vq)
>> used_idx = __atomic_load_n(&(vq)->vq_split.ring.used->idx, \
>> __ATOMIC_RELAXED); \
>> nused = (uint16_t)(used_idx - (vq)->vq_used_cons_idx); \
>> - if (vtpci_packed_queue((vq)->hw)) { \
>> + if (virtio_with_packed_queue((vq)->hw)) { \
>> PMD_INIT_LOG(DEBUG, \
>> "VQ: - size=%d; free=%d; used_cons_idx=%d; avail_idx=%d;" \
>> " cached_flags=0x%x; used_wrap_counter=%d", \
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in generic header
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (16 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 17/40] net/virtio: move features definition to generic header Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:14 ` Xia, Chenbo
2021-01-06 15:53 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to " Maxime Coquelin
` (22 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves the virtqueues defines from PCI header
to the genreric one.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio.h | 18 ++++++++++++++++++
drivers/net/virtio/virtio_ethdev.h | 3 ++-
drivers/net/virtio/virtio_pci.h | 17 -----------------
.../net/virtio/virtio_user/virtio_user_dev.h | 2 +-
4 files changed, 21 insertions(+), 19 deletions(-)
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
index 9e787803a4..eeeb5dba4f 100644
--- a/drivers/net/virtio/virtio.h
+++ b/drivers/net/virtio/virtio.h
@@ -88,6 +88,24 @@
#define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
#define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
+/*
+ * Each virtqueue indirect descriptor list must be physically contiguous.
+ * To allow us to malloc(9) each list individually, limit the number
+ * supported to what will fit in one page. With 4KB pages, this is a limit
+ * of 256 descriptors. If there is ever a need for more, we can switch to
+ * contigmalloc(9) for the larger allocations, similar to what
+ * bus_dmamem_alloc(9) does.
+ *
+ * Note the sizeof(struct vring_desc) is 16 bytes.
+ */
+#define VIRTIO_MAX_INDIRECT ((int)(PAGE_SIZE / 16))
+
+/*
+ * Maximum number of virtqueues per device.
+ */
+#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
+#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
+
struct virtio_hw {
struct virtqueue **vqs;
uint64_t guest_features;
diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
index 13395937c8..6fc373f484 100644
--- a/drivers/net/virtio/virtio_ethdev.h
+++ b/drivers/net/virtio/virtio_ethdev.h
@@ -7,7 +7,8 @@
#include <stdint.h>
-#include "virtio_pci.h"
+#include "virtio.h"
+#include <rte_ethdev_driver.h>
#ifndef PAGE_SIZE
#define PAGE_SIZE 4096
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index b02e5c15f5..249f9754cc 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -67,23 +67,6 @@ struct virtnet_ctl;
#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
#define VIRTIO_CONFIG_STATUS_FAILED 0x80
-/*
- * Each virtqueue indirect descriptor list must be physically contiguous.
- * To allow us to malloc(9) each list individually, limit the number
- * supported to what will fit in one page. With 4KB pages, this is a limit
- * of 256 descriptors. If there is ever a need for more, we can switch to
- * contigmalloc(9) for the larger allocations, similar to what
- * bus_dmamem_alloc(9) does.
- *
- * Note the sizeof(struct vring_desc) is 16 bytes.
- */
-#define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
-
-/*
- * Maximum number of virtqueues per device.
- */
-#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
-#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
/* Common configuration */
#define VIRTIO_PCI_CAP_COMMON_CFG 1
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 59f4dd1f24..0eb481ae21 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -7,7 +7,7 @@
#include <limits.h>
#include <stdbool.h>
-#include "../virtio_pci.h"
+#include "../virtio.h"
#include "../virtio_ring.h"
enum virtio_user_backend_type {
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in " Maxime Coquelin
@ 2020-12-30 3:14 ` Xia, Chenbo
2021-01-06 15:53 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:14 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 18/40] net/virtio: move virtqueue defines in generic header
>
> This patch moves the virtqueues defines from PCI header
> to the genreric one.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.h | 18 ++++++++++++++++++
> drivers/net/virtio/virtio_ethdev.h | 3 ++-
> drivers/net/virtio/virtio_pci.h | 17 -----------------
> .../net/virtio/virtio_user/virtio_user_dev.h | 2 +-
> 4 files changed, 21 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> index 9e787803a4..eeeb5dba4f 100644
> --- a/drivers/net/virtio/virtio.h
> +++ b/drivers/net/virtio/virtio.h
> @@ -88,6 +88,24 @@
> #define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
> #define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
>
> +/*
> + * Each virtqueue indirect descriptor list must be physically contiguous.
> + * To allow us to malloc(9) each list individually, limit the number
> + * supported to what will fit in one page. With 4KB pages, this is a limit
> + * of 256 descriptors. If there is ever a need for more, we can switch to
> + * contigmalloc(9) for the larger allocations, similar to what
> + * bus_dmamem_alloc(9) does.
> + *
> + * Note the sizeof(struct vring_desc) is 16 bytes.
> + */
> +#define VIRTIO_MAX_INDIRECT ((int)(PAGE_SIZE / 16))
> +
> +/*
> + * Maximum number of virtqueues per device.
> + */
> +#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> +#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
> +
> struct virtio_hw {
> struct virtqueue **vqs;
> uint64_t guest_features;
> diff --git a/drivers/net/virtio/virtio_ethdev.h
> b/drivers/net/virtio/virtio_ethdev.h
> index 13395937c8..6fc373f484 100644
> --- a/drivers/net/virtio/virtio_ethdev.h
> +++ b/drivers/net/virtio/virtio_ethdev.h
> @@ -7,7 +7,8 @@
>
> #include <stdint.h>
>
> -#include "virtio_pci.h"
> +#include "virtio.h"
> +#include <rte_ethdev_driver.h>
>
> #ifndef PAGE_SIZE
> #define PAGE_SIZE 4096
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index b02e5c15f5..249f9754cc 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -67,23 +67,6 @@ struct virtnet_ctl;
> #define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
> #define VIRTIO_CONFIG_STATUS_FAILED 0x80
>
> -/*
> - * Each virtqueue indirect descriptor list must be physically contiguous.
> - * To allow us to malloc(9) each list individually, limit the number
> - * supported to what will fit in one page. With 4KB pages, this is a limit
> - * of 256 descriptors. If there is ever a need for more, we can switch to
> - * contigmalloc(9) for the larger allocations, similar to what
> - * bus_dmamem_alloc(9) does.
> - *
> - * Note the sizeof(struct vring_desc) is 16 bytes.
> - */
> -#define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
> -
> -/*
> - * Maximum number of virtqueues per device.
> - */
> -#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> -#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>
> /* Common configuration */
> #define VIRTIO_PCI_CAP_COMMON_CFG 1
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 59f4dd1f24..0eb481ae21 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -7,7 +7,7 @@
>
> #include <limits.h>
> #include <stdbool.h>
> -#include "../virtio_pci.h"
> +#include "../virtio.h"
> #include "../virtio_ring.h"
>
> enum virtio_user_backend_type {
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in " Maxime Coquelin
2020-12-30 3:14 ` Xia, Chenbo
@ 2021-01-06 15:53 ` David Marchand
2021-01-15 10:55 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 15:53 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves the virtqueues defines from PCI header
> to the genreric one.
generic*
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.h | 18 ++++++++++++++++++
> drivers/net/virtio/virtio_ethdev.h | 3 ++-
> drivers/net/virtio/virtio_pci.h | 17 -----------------
> .../net/virtio/virtio_user/virtio_user_dev.h | 2 +-
> 4 files changed, 21 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> index 9e787803a4..eeeb5dba4f 100644
> --- a/drivers/net/virtio/virtio.h
> +++ b/drivers/net/virtio/virtio.h
> @@ -88,6 +88,24 @@
> #define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
> #define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
>
> +/*
> + * Each virtqueue indirect descriptor list must be physically contiguous.
> + * To allow us to malloc(9) each list individually, limit the number
> + * supported to what will fit in one page. With 4KB pages, this is a limit
> + * of 256 descriptors. If there is ever a need for more, we can switch to
> + * contigmalloc(9) for the larger allocations, similar to what
> + * bus_dmamem_alloc(9) does.
> + *
> + * Note the sizeof(struct vring_desc) is 16 bytes.
> + */
> +#define VIRTIO_MAX_INDIRECT ((int)(PAGE_SIZE / 16))
> +
> +/*
> + * Maximum number of virtqueues per device.
> + */
> +#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> +#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
> +
> struct virtio_hw {
> struct virtqueue **vqs;
> uint64_t guest_features;
> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
> index 13395937c8..6fc373f484 100644
> --- a/drivers/net/virtio/virtio_ethdev.h
> +++ b/drivers/net/virtio/virtio_ethdev.h
> @@ -7,7 +7,8 @@
>
> #include <stdint.h>
>
> -#include "virtio_pci.h"
> +#include "virtio.h"
> +#include <rte_ethdev_driver.h>
The coding style recommends group headers inclusion: first system
headers, then dpdk shared headers and finally "local" headers.
https://doc.dpdk.org/guides/contributing/coding_style.html#header-includes
>
> #ifndef PAGE_SIZE
> #define PAGE_SIZE 4096
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index b02e5c15f5..249f9754cc 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -67,23 +67,6 @@ struct virtnet_ctl;
> #define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
> #define VIRTIO_CONFIG_STATUS_FAILED 0x80
>
> -/*
> - * Each virtqueue indirect descriptor list must be physically contiguous.
> - * To allow us to malloc(9) each list individually, limit the number
> - * supported to what will fit in one page. With 4KB pages, this is a limit
> - * of 256 descriptors. If there is ever a need for more, we can switch to
> - * contigmalloc(9) for the larger allocations, similar to what
> - * bus_dmamem_alloc(9) does.
> - *
> - * Note the sizeof(struct vring_desc) is 16 bytes.
> - */
> -#define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
> -
> -/*
> - * Maximum number of virtqueues per device.
> - */
> -#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> -#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>
> /* Common configuration */
> #define VIRTIO_PCI_CAP_COMMON_CFG 1
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 59f4dd1f24..0eb481ae21 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -7,7 +7,7 @@
>
> #include <limits.h>
> #include <stdbool.h>
Same comment, for readability, add an empty line here.
> -#include "../virtio_pci.h"
> +#include "../virtio.h"
> #include "../virtio_ring.h"
>
> enum virtio_user_backend_type {
> --
> 2.29.2
>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in generic header
2021-01-06 15:53 ` David Marchand
@ 2021-01-15 10:55 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:55 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 4:53 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch moves the virtqueues defines from PCI header
>> to the genreric one.
>
>
> generic*
>
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio.h | 18 ++++++++++++++++++
>> drivers/net/virtio/virtio_ethdev.h | 3 ++-
>> drivers/net/virtio/virtio_pci.h | 17 -----------------
>> .../net/virtio/virtio_user/virtio_user_dev.h | 2 +-
>> 4 files changed, 21 insertions(+), 19 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
>> index 9e787803a4..eeeb5dba4f 100644
>> --- a/drivers/net/virtio/virtio.h
>> +++ b/drivers/net/virtio/virtio.h
>> @@ -88,6 +88,24 @@
>> #define VIRTIO_NET_S_LINK_UP 1 /* Link is up */
>> #define VIRTIO_NET_S_ANNOUNCE 2 /* Announcement is needed */
>>
>> +/*
>> + * Each virtqueue indirect descriptor list must be physically contiguous.
>> + * To allow us to malloc(9) each list individually, limit the number
>> + * supported to what will fit in one page. With 4KB pages, this is a limit
>> + * of 256 descriptors. If there is ever a need for more, we can switch to
>> + * contigmalloc(9) for the larger allocations, similar to what
>> + * bus_dmamem_alloc(9) does.
>> + *
>> + * Note the sizeof(struct vring_desc) is 16 bytes.
>> + */
>> +#define VIRTIO_MAX_INDIRECT ((int)(PAGE_SIZE / 16))
>> +
>> +/*
>> + * Maximum number of virtqueues per device.
>> + */
>> +#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
>> +#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>> +
>> struct virtio_hw {
>> struct virtqueue **vqs;
>> uint64_t guest_features;
>> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
>> index 13395937c8..6fc373f484 100644
>> --- a/drivers/net/virtio/virtio_ethdev.h
>> +++ b/drivers/net/virtio/virtio_ethdev.h
>> @@ -7,7 +7,8 @@
>>
>> #include <stdint.h>
>>
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> +#include <rte_ethdev_driver.h>
>
> The coding style recommends group headers inclusion: first system
> headers, then dpdk shared headers and finally "local" headers.
> https://doc.dpdk.org/guides/contributing/coding_style.html#header-includes
>
>
>>
>> #ifndef PAGE_SIZE
>> #define PAGE_SIZE 4096
>> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
>> index b02e5c15f5..249f9754cc 100644
>> --- a/drivers/net/virtio/virtio_pci.h
>> +++ b/drivers/net/virtio/virtio_pci.h
>> @@ -67,23 +67,6 @@ struct virtnet_ctl;
>> #define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
>> #define VIRTIO_CONFIG_STATUS_FAILED 0x80
>>
>> -/*
>> - * Each virtqueue indirect descriptor list must be physically contiguous.
>> - * To allow us to malloc(9) each list individually, limit the number
>> - * supported to what will fit in one page. With 4KB pages, this is a limit
>> - * of 256 descriptors. If there is ever a need for more, we can switch to
>> - * contigmalloc(9) for the larger allocations, similar to what
>> - * bus_dmamem_alloc(9) does.
>> - *
>> - * Note the sizeof(struct vring_desc) is 16 bytes.
>> - */
>> -#define VIRTIO_MAX_INDIRECT ((int) (PAGE_SIZE / 16))
>> -
>> -/*
>> - * Maximum number of virtqueues per device.
>> - */
>> -#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
>> -#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>>
>> /* Common configuration */
>> #define VIRTIO_PCI_CAP_COMMON_CFG 1
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> index 59f4dd1f24..0eb481ae21 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> @@ -7,7 +7,7 @@
>>
>> #include <limits.h>
>> #include <stdbool.h>
>
> Same comment, for readability, add an empty line here.
Fixed it, and also above one.
Thanks,
Maxime
>> -#include "../virtio_pci.h"
>> +#include "../virtio.h"
>> #include "../virtio_ring.h"
>>
>> enum virtio_user_backend_type {
>> --
>> 2.29.2
>>
>
>
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to generic header
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (17 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 18/40] net/virtio: move virtqueue defines in " Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:15 ` Xia, Chenbo
2021-01-06 16:01 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic Maxime Coquelin
` (21 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves config and status definitions from the PCI
header to the generic one.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio.c | 43 +++++++++++++++++++
drivers/net/virtio/virtio.h | 52 ++++++++++++++++++++++-
drivers/net/virtio/virtio_ethdev.c | 36 ++++++++--------
drivers/net/virtio/virtio_pci.c | 44 --------------------
drivers/net/virtio/virtio_pci.h | 55 -------------------------
drivers/net/virtio/virtio_user_ethdev.c | 12 +++---
6 files changed, 118 insertions(+), 124 deletions(-)
diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
index d8d6bf7add..ba3203e68b 100644
--- a/drivers/net/virtio/virtio.c
+++ b/drivers/net/virtio/virtio.c
@@ -20,3 +20,46 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features)
return features;
}
+
+void
+virtio_read_dev_config(struct virtio_hw *hw, size_t offset,
+ void *dst, int length)
+{
+ VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
+}
+
+void
+virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
+ const void *src, int length)
+{
+ VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
+}
+
+void
+virtio_reset(struct virtio_hw *hw)
+{
+ VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
+ /* flush status write */
+ VIRTIO_OPS(hw)->get_status(hw);
+}
+
+void
+virtio_reinit_complete(struct virtio_hw *hw)
+{
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
+}
+
+void
+virtio_set_status(struct virtio_hw *hw, uint8_t status)
+{
+ if (status != VIRTIO_CONFIG_STATUS_RESET)
+ status |= VIRTIO_OPS(hw)->get_status(hw);
+
+ VIRTIO_OPS(hw)->set_status(hw, status);
+}
+
+uint8_t
+virtio_get_status(struct virtio_hw *hw)
+{
+ return VIRTIO_OPS(hw)->get_status(hw);
+}
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
index eeeb5dba4f..5169436c9f 100644
--- a/drivers/net/virtio/virtio.h
+++ b/drivers/net/virtio/virtio.h
@@ -106,6 +106,50 @@
#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
#define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
+/* VirtIO device IDs. */
+#define VIRTIO_ID_NETWORK 0x01
+#define VIRTIO_ID_BLOCK 0x02
+#define VIRTIO_ID_CONSOLE 0x03
+#define VIRTIO_ID_ENTROPY 0x04
+#define VIRTIO_ID_BALLOON 0x05
+#define VIRTIO_ID_IOMEMORY 0x06
+#define VIRTIO_ID_9P 0x09
+
+/* Status byte for guest to report progress. */
+#define VIRTIO_CONFIG_STATUS_RESET 0x00
+#define VIRTIO_CONFIG_STATUS_ACK 0x01
+#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
+#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
+#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
+#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
+#define VIRTIO_CONFIG_STATUS_FAILED 0x80
+
+/*
+ * This structure is just a reference to read
+ * net device specific config space; it just a chodu structure
+ *
+ */
+struct virtio_net_config {
+ /* The config defining mac address (if VIRTIO_NET_F_MAC) */
+ uint8_t mac[RTE_ETHER_ADDR_LEN];
+ /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
+ uint16_t status;
+ uint16_t max_virtqueue_pairs;
+ uint16_t mtu;
+ /*
+ * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
+ * Any other value stands for unknown.
+ */
+ uint32_t speed;
+ /*
+ * 0x00 - half duplex
+ * 0x01 - full duplex
+ * Any other value stands for unknown.
+ */
+ uint8_t duplex;
+
+} __rte_packed;
+
struct virtio_hw {
struct virtqueue **vqs;
uint64_t guest_features;
@@ -159,7 +203,7 @@ struct virtio_ops {
/*
* While virtio_hw is stored in shared memory, this structure stores
* some infos that may vary in the multiple process model locally.
- * For example, the vtpci_ops pointer.
+ * For example, the virtio_ops pointer.
*/
struct virtio_hw_internal {
const struct virtio_ops *virtio_ops;
@@ -183,5 +227,11 @@ virtio_with_packed_queue(struct virtio_hw *hw)
}
uint64_t virtio_negotiate_features(struct virtio_hw *hw, uint64_t host_features);
+uint8_t virtio_get_status(struct virtio_hw *hw);
+void virtio_set_status(struct virtio_hw *hw, uint8_t status);
+void virtio_write_dev_config(struct virtio_hw *hw, size_t offset, const void *src, int length);
+void virtio_read_dev_config(struct virtio_hw *hw, size_t offset, void *dst, int length);
+void virtio_reset(struct virtio_hw *hw);
+void virtio_reinit_complete(struct virtio_hw *hw);
#endif /* _VIRTIO_H_ */
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c9085f2f35..560647f11b 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -714,7 +714,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
dev->intr_handle->intr_vec = NULL;
}
- vtpci_reset(hw);
+ virtio_reset(hw);
virtio_dev_free_mbufs(dev);
virtio_free_queues(hw);
@@ -1096,7 +1096,7 @@ virtio_dev_stats_reset(struct rte_eth_dev *dev)
static void
virtio_set_hwaddr(struct virtio_hw *hw)
{
- vtpci_write_dev_config(hw,
+ virtio_write_dev_config(hw,
offsetof(struct virtio_net_config, mac),
&hw->mac_addr, RTE_ETHER_ADDR_LEN);
}
@@ -1105,7 +1105,7 @@ static void
virtio_get_hwaddr(struct virtio_hw *hw)
{
if (virtio_with_feature(hw, VIRTIO_NET_F_MAC)) {
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, mac),
&hw->mac_addr, RTE_ETHER_ADDR_LEN);
} else {
@@ -1298,7 +1298,7 @@ virtio_ethdev_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
if (host_features & req_features & (1ULL << VIRTIO_NET_F_MTU)) {
struct virtio_net_config config;
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, mtu),
&config.mtu, sizeof(config.mtu));
@@ -1321,9 +1321,9 @@ virtio_ethdev_negotiate_features(struct virtio_hw *hw, uint64_t req_features)
}
if (virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
- if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
+ if (!(virtio_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
return -1;
}
@@ -1455,7 +1455,7 @@ virtio_interrupt_handler(void *param)
NULL);
if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
&status, sizeof(status));
if (status & VIRTIO_NET_S_ANNOUNCE) {
@@ -1637,7 +1637,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
int ret;
/* Reset the device although not necessary at startup */
- vtpci_reset(hw);
+ virtio_reset(hw);
if (hw->vqs) {
virtio_dev_free_mbufs(eth_dev);
@@ -1645,10 +1645,10 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
}
/* Tell the host we've noticed this device. */
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
/* Tell the host we've known how to drive the device. */
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
if (virtio_ethdev_negotiate_features(hw, req_features) < 0)
return -1;
@@ -1683,10 +1683,10 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
if (hw->speed == ETH_SPEED_NUM_UNKNOWN) {
if (virtio_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
config = &local_config;
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, speed),
&config->speed, sizeof(config->speed));
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, duplex),
&config->duplex, sizeof(config->duplex));
hw->speed = config->speed;
@@ -1700,12 +1700,12 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
config = &local_config;
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, mac),
&config->mac, sizeof(config->mac));
if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
&config->status, sizeof(config->status));
} else {
@@ -1715,7 +1715,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
}
if (virtio_with_feature(hw, VIRTIO_NET_F_MQ)) {
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, max_virtqueue_pairs),
&config->max_virtqueue_pairs,
sizeof(config->max_virtqueue_pairs));
@@ -1728,7 +1728,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->max_queue_pairs = config->max_virtqueue_pairs;
if (virtio_with_feature(hw, VIRTIO_NET_F_MTU)) {
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, mtu),
&config->mtu,
sizeof(config->mtu));
@@ -1780,7 +1780,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
}
}
- vtpci_reinit_complete(hw);
+ virtio_reinit_complete(hw);
return 0;
}
@@ -2352,7 +2352,7 @@ virtio_dev_link_update(struct rte_eth_dev *dev, __rte_unused int wait_to_complet
link.link_speed = ETH_SPEED_NUM_NONE;
} else if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
PMD_INIT_LOG(DEBUG, "Get link status from hw");
- vtpci_read_dev_config(hw,
+ virtio_read_dev_config(hw,
offsetof(struct virtio_net_config, status),
&status, sizeof(status));
if ((status & VIRTIO_NET_S_LINK_UP) == 0) {
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9c07ebad00..29dd84888f 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -533,50 +533,6 @@ const struct virtio_ops modern_ops = {
.dev_close = modern_dev_close,
};
-
-void
-vtpci_read_dev_config(struct virtio_hw *hw, size_t offset,
- void *dst, int length)
-{
- VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
-}
-
-void
-vtpci_write_dev_config(struct virtio_hw *hw, size_t offset,
- const void *src, int length)
-{
- VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
-}
-
-void
-vtpci_reset(struct virtio_hw *hw)
-{
- VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
- /* flush status write */
- VIRTIO_OPS(hw)->get_status(hw);
-}
-
-void
-vtpci_reinit_complete(struct virtio_hw *hw)
-{
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
-}
-
-void
-vtpci_set_status(struct virtio_hw *hw, uint8_t status)
-{
- if (status != VIRTIO_CONFIG_STATUS_RESET)
- status |= VIRTIO_OPS(hw)->get_status(hw);
-
- VIRTIO_OPS(hw)->set_status(hw, status);
-}
-
-uint8_t
-vtpci_get_status(struct virtio_hw *hw)
-{
- return VIRTIO_OPS(hw)->get_status(hw);
-}
-
uint8_t
vtpci_isr(struct virtio_hw *hw)
{
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 249f9754cc..19d56c920c 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -49,25 +49,6 @@ struct virtnet_ctl;
/* Vector value used to disable MSI for queue. */
#define VIRTIO_MSI_NO_VECTOR 0xFFFF
-/* VirtIO device IDs. */
-#define VIRTIO_ID_NETWORK 0x01
-#define VIRTIO_ID_BLOCK 0x02
-#define VIRTIO_ID_CONSOLE 0x03
-#define VIRTIO_ID_ENTROPY 0x04
-#define VIRTIO_ID_BALLOON 0x05
-#define VIRTIO_ID_IOMEMORY 0x06
-#define VIRTIO_ID_9P 0x09
-
-/* Status byte for guest to report progress. */
-#define VIRTIO_CONFIG_STATUS_RESET 0x00
-#define VIRTIO_CONFIG_STATUS_ACK 0x01
-#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
-#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
-#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
-#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
-#define VIRTIO_CONFIG_STATUS_FAILED 0x80
-
-
/* Common configuration */
#define VIRTIO_PCI_CAP_COMMON_CFG 1
/* Notifications */
@@ -136,32 +117,6 @@ struct virtio_pci_dev {
#define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
-/*
- * This structure is just a reference to read
- * net device specific config space; it just a chodu structure
- *
- */
-struct virtio_net_config {
- /* The config defining mac address (if VIRTIO_NET_F_MAC) */
- uint8_t mac[RTE_ETHER_ADDR_LEN];
- /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
- uint16_t status;
- uint16_t max_virtqueue_pairs;
- uint16_t mtu;
- /*
- * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
- * Any other value stands for unknown.
- */
- uint32_t speed;
- /*
- * 0x00 - half duplex
- * 0x01 - full duplex
- * Any other value stands for unknown.
- */
- uint8_t duplex;
-
-} __rte_packed;
-
/*
* How many bits to shift physical queue address written to QUEUE_PFN.
* 12 is historical, and due to x86 page size.
@@ -182,16 +137,6 @@ enum virtio_msix_status {
* Function declaration from virtio_pci.c
*/
int vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev);
-void vtpci_reset(struct virtio_hw *);
-
-void vtpci_reinit_complete(struct virtio_hw *);
-
-uint8_t vtpci_get_status(struct virtio_hw *);
-void vtpci_set_status(struct virtio_hw *, uint8_t);
-
-void vtpci_write_dev_config(struct virtio_hw *, size_t, const void *, int);
-
-void vtpci_read_dev_config(struct virtio_hw *, size_t, void *, int);
uint8_t vtpci_isr(struct virtio_hw *);
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index d05613ba3b..0f252c0732 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -77,13 +77,13 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
return -1;
dev->vhostfd = connectfd;
- old_status = vtpci_get_status(hw);
+ old_status = virtio_get_status(hw);
- vtpci_reset(hw);
+ virtio_reset(hw);
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
&dev->device_features) < 0) {
@@ -129,10 +129,10 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
virtio_user_reset_queues_packed(eth_dev);
}
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
/* Start the device */
- vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
if (!dev->started)
return -1;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to " Maxime Coquelin
@ 2020-12-30 3:15 ` Xia, Chenbo
2021-01-06 16:01 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:15 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 19/40] net/virtio: move config definitions to generic header
>
> This patch moves config and status definitions from the PCI
> header to the generic one.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.c | 43 +++++++++++++++++++
> drivers/net/virtio/virtio.h | 52 ++++++++++++++++++++++-
> drivers/net/virtio/virtio_ethdev.c | 36 ++++++++--------
> drivers/net/virtio/virtio_pci.c | 44 --------------------
> drivers/net/virtio/virtio_pci.h | 55 -------------------------
> drivers/net/virtio/virtio_user_ethdev.c | 12 +++---
> 6 files changed, 118 insertions(+), 124 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
> index d8d6bf7add..ba3203e68b 100644
> --- a/drivers/net/virtio/virtio.c
> +++ b/drivers/net/virtio/virtio.c
> @@ -20,3 +20,46 @@ virtio_negotiate_features(struct virtio_hw *hw, uint64_t
> host_features)
> return features;
> }
>
> +
> +void
> +virtio_read_dev_config(struct virtio_hw *hw, size_t offset,
> + void *dst, int length)
> +{
> + VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
> +}
> +
> +void
> +virtio_write_dev_config(struct virtio_hw *hw, size_t offset,
> + const void *src, int length)
> +{
> + VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
> +}
> +
> +void
> +virtio_reset(struct virtio_hw *hw)
> +{
> + VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
> + /* flush status write */
> + VIRTIO_OPS(hw)->get_status(hw);
> +}
> +
> +void
> +virtio_reinit_complete(struct virtio_hw *hw)
> +{
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
> +}
> +
> +void
> +virtio_set_status(struct virtio_hw *hw, uint8_t status)
> +{
> + if (status != VIRTIO_CONFIG_STATUS_RESET)
> + status |= VIRTIO_OPS(hw)->get_status(hw);
> +
> + VIRTIO_OPS(hw)->set_status(hw, status);
> +}
> +
> +uint8_t
> +virtio_get_status(struct virtio_hw *hw)
> +{
> + return VIRTIO_OPS(hw)->get_status(hw);
> +}
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> index eeeb5dba4f..5169436c9f 100644
> --- a/drivers/net/virtio/virtio.h
> +++ b/drivers/net/virtio/virtio.h
> @@ -106,6 +106,50 @@
> #define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> #define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>
> +/* VirtIO device IDs. */
> +#define VIRTIO_ID_NETWORK 0x01
> +#define VIRTIO_ID_BLOCK 0x02
> +#define VIRTIO_ID_CONSOLE 0x03
> +#define VIRTIO_ID_ENTROPY 0x04
> +#define VIRTIO_ID_BALLOON 0x05
> +#define VIRTIO_ID_IOMEMORY 0x06
> +#define VIRTIO_ID_9P 0x09
> +
> +/* Status byte for guest to report progress. */
> +#define VIRTIO_CONFIG_STATUS_RESET 0x00
> +#define VIRTIO_CONFIG_STATUS_ACK 0x01
> +#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
> +#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
> +#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
> +#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
> +#define VIRTIO_CONFIG_STATUS_FAILED 0x80
> +
> +/*
> + * This structure is just a reference to read
> + * net device specific config space; it just a chodu structure
> + *
> + */
> +struct virtio_net_config {
> + /* The config defining mac address (if VIRTIO_NET_F_MAC) */
> + uint8_t mac[RTE_ETHER_ADDR_LEN];
> + /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
> + uint16_t status;
> + uint16_t max_virtqueue_pairs;
> + uint16_t mtu;
> + /*
> + * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
> + * Any other value stands for unknown.
> + */
> + uint32_t speed;
> + /*
> + * 0x00 - half duplex
> + * 0x01 - full duplex
> + * Any other value stands for unknown.
> + */
> + uint8_t duplex;
> +
> +} __rte_packed;
> +
> struct virtio_hw {
> struct virtqueue **vqs;
> uint64_t guest_features;
> @@ -159,7 +203,7 @@ struct virtio_ops {
> /*
> * While virtio_hw is stored in shared memory, this structure stores
> * some infos that may vary in the multiple process model locally.
> - * For example, the vtpci_ops pointer.
> + * For example, the virtio_ops pointer.
> */
> struct virtio_hw_internal {
> const struct virtio_ops *virtio_ops;
> @@ -183,5 +227,11 @@ virtio_with_packed_queue(struct virtio_hw *hw)
> }
>
> uint64_t virtio_negotiate_features(struct virtio_hw *hw, uint64_t
> host_features);
> +uint8_t virtio_get_status(struct virtio_hw *hw);
> +void virtio_set_status(struct virtio_hw *hw, uint8_t status);
> +void virtio_write_dev_config(struct virtio_hw *hw, size_t offset, const void
> *src, int length);
> +void virtio_read_dev_config(struct virtio_hw *hw, size_t offset, void *dst,
> int length);
> +void virtio_reset(struct virtio_hw *hw);
> +void virtio_reinit_complete(struct virtio_hw *hw);
>
> #endif /* _VIRTIO_H_ */
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index c9085f2f35..560647f11b 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -714,7 +714,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
> dev->intr_handle->intr_vec = NULL;
> }
>
> - vtpci_reset(hw);
> + virtio_reset(hw);
> virtio_dev_free_mbufs(dev);
> virtio_free_queues(hw);
>
> @@ -1096,7 +1096,7 @@ virtio_dev_stats_reset(struct rte_eth_dev *dev)
> static void
> virtio_set_hwaddr(struct virtio_hw *hw)
> {
> - vtpci_write_dev_config(hw,
> + virtio_write_dev_config(hw,
> offsetof(struct virtio_net_config, mac),
> &hw->mac_addr, RTE_ETHER_ADDR_LEN);
> }
> @@ -1105,7 +1105,7 @@ static void
> virtio_get_hwaddr(struct virtio_hw *hw)
> {
> if (virtio_with_feature(hw, VIRTIO_NET_F_MAC)) {
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, mac),
> &hw->mac_addr, RTE_ETHER_ADDR_LEN);
> } else {
> @@ -1298,7 +1298,7 @@ virtio_ethdev_negotiate_features(struct virtio_hw *hw,
> uint64_t req_features)
> if (host_features & req_features & (1ULL << VIRTIO_NET_F_MTU)) {
> struct virtio_net_config config;
>
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, mtu),
> &config.mtu, sizeof(config.mtu));
>
> @@ -1321,9 +1321,9 @@ virtio_ethdev_negotiate_features(struct virtio_hw *hw,
> uint64_t req_features)
> }
>
> if (virtio_with_feature(hw, VIRTIO_F_VERSION_1)) {
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
>
> - if (!(vtpci_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
> + if (!(virtio_get_status(hw) & VIRTIO_CONFIG_STATUS_FEATURES_OK)) {
> PMD_INIT_LOG(ERR, "Failed to set FEATURES_OK status!");
> return -1;
> }
> @@ -1455,7 +1455,7 @@ virtio_interrupt_handler(void *param)
> NULL);
>
> if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, status),
> &status, sizeof(status));
> if (status & VIRTIO_NET_S_ANNOUNCE) {
> @@ -1637,7 +1637,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> int ret;
>
> /* Reset the device although not necessary at startup */
> - vtpci_reset(hw);
> + virtio_reset(hw);
>
> if (hw->vqs) {
> virtio_dev_free_mbufs(eth_dev);
> @@ -1645,10 +1645,10 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> }
>
> /* Tell the host we've noticed this device. */
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
>
> /* Tell the host we've known how to drive the device. */
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
> if (virtio_ethdev_negotiate_features(hw, req_features) < 0)
> return -1;
>
> @@ -1683,10 +1683,10 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> if (hw->speed == ETH_SPEED_NUM_UNKNOWN) {
> if (virtio_with_feature(hw, VIRTIO_NET_F_SPEED_DUPLEX)) {
> config = &local_config;
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, speed),
> &config->speed, sizeof(config->speed));
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, duplex),
> &config->duplex, sizeof(config->duplex));
> hw->speed = config->speed;
> @@ -1700,12 +1700,12 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> if (virtio_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) {
> config = &local_config;
>
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, mac),
> &config->mac, sizeof(config->mac));
>
> if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, status),
> &config->status, sizeof(config->status));
> } else {
> @@ -1715,7 +1715,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> }
>
> if (virtio_with_feature(hw, VIRTIO_NET_F_MQ)) {
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config,
> max_virtqueue_pairs),
> &config->max_virtqueue_pairs,
> sizeof(config->max_virtqueue_pairs));
> @@ -1728,7 +1728,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> hw->max_queue_pairs = config->max_virtqueue_pairs;
>
> if (virtio_with_feature(hw, VIRTIO_NET_F_MTU)) {
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, mtu),
> &config->mtu,
> sizeof(config->mtu));
> @@ -1780,7 +1780,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t
> req_features)
> }
> }
>
> - vtpci_reinit_complete(hw);
> + virtio_reinit_complete(hw);
>
> return 0;
> }
> @@ -2352,7 +2352,7 @@ virtio_dev_link_update(struct rte_eth_dev *dev,
> __rte_unused int wait_to_complet
> link.link_speed = ETH_SPEED_NUM_NONE;
> } else if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS)) {
> PMD_INIT_LOG(DEBUG, "Get link status from hw");
> - vtpci_read_dev_config(hw,
> + virtio_read_dev_config(hw,
> offsetof(struct virtio_net_config, status),
> &status, sizeof(status));
> if ((status & VIRTIO_NET_S_LINK_UP) == 0) {
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 9c07ebad00..29dd84888f 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -533,50 +533,6 @@ const struct virtio_ops modern_ops = {
> .dev_close = modern_dev_close,
> };
>
> -
> -void
> -vtpci_read_dev_config(struct virtio_hw *hw, size_t offset,
> - void *dst, int length)
> -{
> - VIRTIO_OPS(hw)->read_dev_cfg(hw, offset, dst, length);
> -}
> -
> -void
> -vtpci_write_dev_config(struct virtio_hw *hw, size_t offset,
> - const void *src, int length)
> -{
> - VIRTIO_OPS(hw)->write_dev_cfg(hw, offset, src, length);
> -}
> -
> -void
> -vtpci_reset(struct virtio_hw *hw)
> -{
> - VIRTIO_OPS(hw)->set_status(hw, VIRTIO_CONFIG_STATUS_RESET);
> - /* flush status write */
> - VIRTIO_OPS(hw)->get_status(hw);
> -}
> -
> -void
> -vtpci_reinit_complete(struct virtio_hw *hw)
> -{
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
> -}
> -
> -void
> -vtpci_set_status(struct virtio_hw *hw, uint8_t status)
> -{
> - if (status != VIRTIO_CONFIG_STATUS_RESET)
> - status |= VIRTIO_OPS(hw)->get_status(hw);
> -
> - VIRTIO_OPS(hw)->set_status(hw, status);
> -}
> -
> -uint8_t
> -vtpci_get_status(struct virtio_hw *hw)
> -{
> - return VIRTIO_OPS(hw)->get_status(hw);
> -}
> -
> uint8_t
> vtpci_isr(struct virtio_hw *hw)
> {
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 249f9754cc..19d56c920c 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -49,25 +49,6 @@ struct virtnet_ctl;
> /* Vector value used to disable MSI for queue. */
> #define VIRTIO_MSI_NO_VECTOR 0xFFFF
>
> -/* VirtIO device IDs. */
> -#define VIRTIO_ID_NETWORK 0x01
> -#define VIRTIO_ID_BLOCK 0x02
> -#define VIRTIO_ID_CONSOLE 0x03
> -#define VIRTIO_ID_ENTROPY 0x04
> -#define VIRTIO_ID_BALLOON 0x05
> -#define VIRTIO_ID_IOMEMORY 0x06
> -#define VIRTIO_ID_9P 0x09
> -
> -/* Status byte for guest to report progress. */
> -#define VIRTIO_CONFIG_STATUS_RESET 0x00
> -#define VIRTIO_CONFIG_STATUS_ACK 0x01
> -#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
> -#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
> -#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
> -#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
> -#define VIRTIO_CONFIG_STATUS_FAILED 0x80
> -
> -
> /* Common configuration */
> #define VIRTIO_PCI_CAP_COMMON_CFG 1
> /* Notifications */
> @@ -136,32 +117,6 @@ struct virtio_pci_dev {
>
> #define virtio_pci_get_dev(hw) container_of(hw, struct virtio_pci_dev, hw)
>
> -/*
> - * This structure is just a reference to read
> - * net device specific config space; it just a chodu structure
> - *
> - */
> -struct virtio_net_config {
> - /* The config defining mac address (if VIRTIO_NET_F_MAC) */
> - uint8_t mac[RTE_ETHER_ADDR_LEN];
> - /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
> - uint16_t status;
> - uint16_t max_virtqueue_pairs;
> - uint16_t mtu;
> - /*
> - * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
> - * Any other value stands for unknown.
> - */
> - uint32_t speed;
> - /*
> - * 0x00 - half duplex
> - * 0x01 - full duplex
> - * Any other value stands for unknown.
> - */
> - uint8_t duplex;
> -
> -} __rte_packed;
> -
> /*
> * How many bits to shift physical queue address written to QUEUE_PFN.
> * 12 is historical, and due to x86 page size.
> @@ -182,16 +137,6 @@ enum virtio_msix_status {
> * Function declaration from virtio_pci.c
> */
> int vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev);
> -void vtpci_reset(struct virtio_hw *);
> -
> -void vtpci_reinit_complete(struct virtio_hw *);
> -
> -uint8_t vtpci_get_status(struct virtio_hw *);
> -void vtpci_set_status(struct virtio_hw *, uint8_t);
> -
> -void vtpci_write_dev_config(struct virtio_hw *, size_t, const void *, int);
> -
> -void vtpci_read_dev_config(struct virtio_hw *, size_t, void *, int);
>
> uint8_t vtpci_isr(struct virtio_hw *);
>
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index d05613ba3b..0f252c0732 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -77,13 +77,13 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
> return -1;
>
> dev->vhostfd = connectfd;
> - old_status = vtpci_get_status(hw);
> + old_status = virtio_get_status(hw);
>
> - vtpci_reset(hw);
> + virtio_reset(hw);
>
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
>
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>
> if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> &dev->device_features) < 0) {
> @@ -129,10 +129,10 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
> virtio_user_reset_queues_packed(eth_dev);
> }
>
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
>
> /* Start the device */
> - vtpci_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
> if (!dev->started)
> return -1;
>
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to " Maxime Coquelin
2020-12-30 3:15 ` Xia, Chenbo
@ 2021-01-06 16:01 ` David Marchand
2021-01-15 11:01 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 16:01 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> index eeeb5dba4f..5169436c9f 100644
> --- a/drivers/net/virtio/virtio.h
> +++ b/drivers/net/virtio/virtio.h
> @@ -106,6 +106,50 @@
> #define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
> #define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>
> +/* VirtIO device IDs. */
> +#define VIRTIO_ID_NETWORK 0x01
> +#define VIRTIO_ID_BLOCK 0x02
> +#define VIRTIO_ID_CONSOLE 0x03
> +#define VIRTIO_ID_ENTROPY 0x04
> +#define VIRTIO_ID_BALLOON 0x05
> +#define VIRTIO_ID_IOMEMORY 0x06
> +#define VIRTIO_ID_9P 0x09
> +
> +/* Status byte for guest to report progress. */
> +#define VIRTIO_CONFIG_STATUS_RESET 0x00
> +#define VIRTIO_CONFIG_STATUS_ACK 0x01
> +#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
> +#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
> +#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
> +#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
> +#define VIRTIO_CONFIG_STATUS_FAILED 0x80
> +
> +/*
> + * This structure is just a reference to read
> + * net device specific config space; it just a chodu structure
chodu ? :-)
> + *
> + */
> +struct virtio_net_config {
> + /* The config defining mac address (if VIRTIO_NET_F_MAC) */
> + uint8_t mac[RTE_ETHER_ADDR_LEN];
> + /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
> + uint16_t status;
> + uint16_t max_virtqueue_pairs;
> + uint16_t mtu;
> + /*
> + * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
> + * Any other value stands for unknown.
> + */
> + uint32_t speed;
> + /*
> + * 0x00 - half duplex
> + * 0x01 - full duplex
> + * Any other value stands for unknown.
> + */
> + uint8_t duplex;
> +
> +} __rte_packed;
> +
> struct virtio_hw {
> struct virtqueue **vqs;
> uint64_t guest_features;
> @@ -159,7 +203,7 @@ struct virtio_ops {
> /*
> * While virtio_hw is stored in shared memory, this structure stores
> * some infos that may vary in the multiple process model locally.
> - * For example, the vtpci_ops pointer.
> + * For example, the virtio_ops pointer.
It should be in a previous patch (I suggested removing this comment earlier).
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to generic header
2021-01-06 16:01 ` David Marchand
@ 2021-01-15 11:01 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 11:01 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 5:01 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
>> index eeeb5dba4f..5169436c9f 100644
>> --- a/drivers/net/virtio/virtio.h
>> +++ b/drivers/net/virtio/virtio.h
>> @@ -106,6 +106,50 @@
>> #define VIRTIO_MAX_VIRTQUEUE_PAIRS 8
>> #define VIRTIO_MAX_VIRTQUEUES (VIRTIO_MAX_VIRTQUEUE_PAIRS * 2 + 1)
>>
>> +/* VirtIO device IDs. */
>> +#define VIRTIO_ID_NETWORK 0x01
>> +#define VIRTIO_ID_BLOCK 0x02
>> +#define VIRTIO_ID_CONSOLE 0x03
>> +#define VIRTIO_ID_ENTROPY 0x04
>> +#define VIRTIO_ID_BALLOON 0x05
>> +#define VIRTIO_ID_IOMEMORY 0x06
>> +#define VIRTIO_ID_9P 0x09
>> +
>> +/* Status byte for guest to report progress. */
>> +#define VIRTIO_CONFIG_STATUS_RESET 0x00
>> +#define VIRTIO_CONFIG_STATUS_ACK 0x01
>> +#define VIRTIO_CONFIG_STATUS_DRIVER 0x02
>> +#define VIRTIO_CONFIG_STATUS_DRIVER_OK 0x04
>> +#define VIRTIO_CONFIG_STATUS_FEATURES_OK 0x08
>> +#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
>> +#define VIRTIO_CONFIG_STATUS_FAILED 0x80
>> +
>> +/*
>> + * This structure is just a reference to read
>> + * net device specific config space; it just a chodu structure
>
> chodu ? :-)
I guess shadow was meant here, given the start of the sentence.
Will fix it.
>> + *
>> + */
>> +struct virtio_net_config {
>> + /* The config defining mac address (if VIRTIO_NET_F_MAC) */
>> + uint8_t mac[RTE_ETHER_ADDR_LEN];
>> + /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
>> + uint16_t status;
>> + uint16_t max_virtqueue_pairs;
>> + uint16_t mtu;
>> + /*
>> + * speed, in units of 1Mb. All values 0 to INT_MAX are legal.
>> + * Any other value stands for unknown.
>> + */
>> + uint32_t speed;
>> + /*
>> + * 0x00 - half duplex
>> + * 0x01 - full duplex
>> + * Any other value stands for unknown.
>> + */
>> + uint8_t duplex;
>> +
>> +} __rte_packed;
>> +
>> struct virtio_hw {
>> struct virtqueue **vqs;
>> uint64_t guest_features;
>> @@ -159,7 +203,7 @@ struct virtio_ops {
>> /*
>> * While virtio_hw is stored in shared memory, this structure stores
>> * some infos that may vary in the multiple process model locally.
>> - * For example, the vtpci_ops pointer.
>> + * For example, the virtio_ops pointer.
>
> It should be in a previous patch (I suggested removing this comment earlier).
Yes, I guess it was squashed here by mistake during a rebase.
I didn't remove the comment
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (18 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 19/40] net/virtio: move config definitions to " Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:17 ` Xia, Chenbo
2021-01-06 16:07 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header Maxime Coquelin
` (20 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch aims at isolating MSIX notion into PCI
layer.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio.c | 6 ++
drivers/net/virtio/virtio.h | 11 +-
drivers/net/virtio/virtio_ethdev.c | 7 +-
drivers/net/virtio/virtio_pci.c | 131 ++++++++++++------------
drivers/net/virtio/virtio_pci.h | 25 ++---
drivers/net/virtio/virtio_user_ethdev.c | 4 +-
6 files changed, 90 insertions(+), 94 deletions(-)
diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
index ba3203e68b..7e1e77797f 100644
--- a/drivers/net/virtio/virtio.c
+++ b/drivers/net/virtio/virtio.c
@@ -63,3 +63,9 @@ virtio_get_status(struct virtio_hw *hw)
{
return VIRTIO_OPS(hw)->get_status(hw);
}
+
+uint8_t
+virtio_get_isr(struct virtio_hw *hw)
+{
+ return VIRTIO_OPS(hw)->get_isr(hw);
+}
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
index 5169436c9f..f44125f48a 100644
--- a/drivers/net/virtio/virtio.h
+++ b/drivers/net/virtio/virtio.h
@@ -124,6 +124,13 @@
#define VIRTIO_CONFIG_STATUS_DEV_NEED_RESET 0x40
#define VIRTIO_CONFIG_STATUS_FAILED 0x80
+/* The bit of the ISR which indicates a device has an interrupt. */
+#define VIRTIO_ISR_INTR 0x1
+/* The bit of the ISR which indicates a device configuration change. */
+#define VIRTIO_ISR_CONFIG 0x2
+/* Vector value used to disable MSI for queue. */
+#define VIRTIO_MSI_NO_VECTOR 0xFFFF
+
/*
* This structure is just a reference to read
* net device specific config space; it just a chodu structure
@@ -168,7 +175,7 @@ struct virtio_hw {
uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
uint32_t speed; /* link speed in MB */
uint8_t duplex;
- uint8_t use_msix;
+ uint8_t intr_lsc;
uint16_t max_mtu;
/*
* App management thread and virtio interrupt handler thread
@@ -233,5 +240,5 @@ void virtio_write_dev_config(struct virtio_hw *hw, size_t offset, const void *sr
void virtio_read_dev_config(struct virtio_hw *hw, size_t offset, void *dst, int length);
void virtio_reset(struct virtio_hw *hw);
void virtio_reinit_complete(struct virtio_hw *hw);
-
+uint8_t virtio_get_isr(struct virtio_hw *hw);
#endif /* _VIRTIO_H_ */
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 560647f11b..99a2dd24c4 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1442,13 +1442,13 @@ virtio_interrupt_handler(void *param)
uint16_t status;
/* Read interrupt status which clears interrupt */
- isr = vtpci_isr(hw);
+ isr = virtio_get_isr(hw);
PMD_DRV_LOG(INFO, "interrupt status = %#x", isr);
if (virtio_intr_unmask(dev) < 0)
PMD_DRV_LOG(ERR, "interrupt enable failed");
- if (isr & VIRTIO_PCI_ISR_CONFIG) {
+ if (isr & VIRTIO_ISR_CONFIG) {
if (virtio_dev_link_update(dev, 0) == 0)
rte_eth_dev_callback_process(dev,
RTE_ETH_EVENT_INTR_LSC,
@@ -1655,8 +1655,7 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
hw->weak_barriers = !virtio_with_feature(hw, VIRTIO_F_ORDER_PLATFORM);
/* If host does not support both status and MSI-X then disable LSC */
- if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS) &&
- hw->use_msix != VIRTIO_MSIX_NONE)
+ if (virtio_with_feature(hw, VIRTIO_NET_F_STATUS) && hw->intr_lsc)
eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
else
eth_dev->data->dev_flags &= ~RTE_ETH_DEV_INTR_LSC;
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 29dd84888f..01a437a1b4 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -28,8 +28,8 @@
* The remaining space is defined by each driver as the per-driver
* configuration space.
*/
-#define VIRTIO_PCI_CONFIG(hw) \
- (((hw)->use_msix == VIRTIO_MSIX_ENABLED) ? 24 : 20)
+#define VIRTIO_PCI_CONFIG(dev) \
+ (((dev)->msix_status == VIRTIO_MSIX_ENABLED) ? 24 : 20)
struct virtio_pci_internal {
@@ -71,6 +71,7 @@ static void
legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
void *dst, int length)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
#ifdef RTE_ARCH_PPC_64
int size;
@@ -78,17 +79,17 @@ legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
if (length >= 4) {
size = 4;
rte_pci_ioport_read(VTPCI_IO(hw), dst, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
*(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
} else if (length >= 2) {
size = 2;
rte_pci_ioport_read(VTPCI_IO(hw), dst, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
*(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
} else {
size = 1;
rte_pci_ioport_read(VTPCI_IO(hw), dst, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
}
dst = (char *)dst + size;
@@ -97,7 +98,7 @@ legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
}
#else
rte_pci_ioport_read(VTPCI_IO(hw), dst, length,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
#endif
}
@@ -105,6 +106,7 @@ static void
legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
const void *src, int length)
{
+ struct virtio_pci_dev *dev = virtio_pci_get_dev(hw);
#ifdef RTE_ARCH_PPC_64
union {
uint32_t u32;
@@ -117,16 +119,16 @@ legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
size = 4;
tmp.u32 = rte_cpu_to_be_32(*(const uint32_t *)src);
rte_pci_ioport_write(VTPCI_IO(hw), &tmp.u32, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
} else if (length >= 2) {
size = 2;
tmp.u16 = rte_cpu_to_be_16(*(const uint16_t *)src);
rte_pci_ioport_write(VTPCI_IO(hw), &tmp.u16, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
} else {
size = 1;
rte_pci_ioport_write(VTPCI_IO(hw), src, size,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
}
src = (const char *)src + size;
@@ -135,7 +137,7 @@ legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
}
#else
rte_pci_ioport_write(VTPCI_IO(hw), src, length,
- VIRTIO_PCI_CONFIG(hw) + offset);
+ VIRTIO_PCI_CONFIG(dev) + offset);
#endif
}
@@ -533,12 +535,6 @@ const struct virtio_ops modern_ops = {
.dev_close = modern_dev_close,
};
-uint8_t
-vtpci_isr(struct virtio_hw *hw)
-{
- return VIRTIO_OPS(hw)->get_isr(hw);
-}
-
static void *
get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap)
{
@@ -623,9 +619,9 @@ virtio_read_caps(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
}
if (flags & PCI_MSIX_ENABLE)
- hw->use_msix = VIRTIO_MSIX_ENABLED;
+ dev->msix_status = VIRTIO_MSIX_ENABLED;
else
- hw->use_msix = VIRTIO_MSIX_DISABLED;
+ dev->msix_status = VIRTIO_MSIX_DISABLED;
}
if (cap.cap_vndr != PCI_CAP_ID_VNDR) {
@@ -691,6 +687,54 @@ virtio_read_caps(struct rte_pci_device *pci_dev, struct virtio_hw *hw)
return 0;
}
+static enum virtio_msix_status
+vtpci_msix_detect(struct rte_pci_device *dev)
+{
+ uint8_t pos;
+ int ret;
+
+ ret = rte_pci_read_config(dev, &pos, 1, PCI_CAPABILITY_LIST);
+ if (ret != 1) {
+ PMD_INIT_LOG(DEBUG,
+ "failed to read pci capability list, ret %d", ret);
+ return VIRTIO_MSIX_NONE;
+ }
+
+ while (pos) {
+ uint8_t cap[2];
+
+ ret = rte_pci_read_config(dev, cap, sizeof(cap), pos);
+ if (ret != sizeof(cap)) {
+ PMD_INIT_LOG(DEBUG,
+ "failed to read pci cap at pos: %x ret %d",
+ pos, ret);
+ break;
+ }
+
+ if (cap[0] == PCI_CAP_ID_MSIX) {
+ uint16_t flags;
+
+ ret = rte_pci_read_config(dev, &flags, sizeof(flags),
+ pos + sizeof(cap));
+ if (ret != sizeof(flags)) {
+ PMD_INIT_LOG(DEBUG,
+ "failed to read pci cap at pos:"
+ " %x ret %d", pos + 2, ret);
+ break;
+ }
+
+ if (flags & PCI_MSIX_ENABLE)
+ return VIRTIO_MSIX_ENABLED;
+ else
+ return VIRTIO_MSIX_DISABLED;
+ }
+
+ pos = cap[1];
+ }
+
+ return VIRTIO_MSIX_NONE;
+}
+
/*
* Return -1:
* if there is error mapping with VFIO/UIO.
@@ -736,59 +780,12 @@ vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev)
dev->modern = false;
msix_detect:
- hw->use_msix = vtpci_msix_detect(pci_dev);
+ dev->msix_status = vtpci_msix_detect(pci_dev);
+ hw->intr_lsc = !!dev->msix_status;
return 0;
}
-enum virtio_msix_status
-vtpci_msix_detect(struct rte_pci_device *dev)
-{
- uint8_t pos;
- int ret;
-
- ret = rte_pci_read_config(dev, &pos, 1, PCI_CAPABILITY_LIST);
- if (ret != 1) {
- PMD_INIT_LOG(DEBUG,
- "failed to read pci capability list, ret %d", ret);
- return VIRTIO_MSIX_NONE;
- }
-
- while (pos) {
- uint8_t cap[2];
-
- ret = rte_pci_read_config(dev, cap, sizeof(cap), pos);
- if (ret != sizeof(cap)) {
- PMD_INIT_LOG(DEBUG,
- "failed to read pci cap at pos: %x ret %d",
- pos, ret);
- break;
- }
-
- if (cap[0] == PCI_CAP_ID_MSIX) {
- uint16_t flags;
-
- ret = rte_pci_read_config(dev, &flags, sizeof(flags),
- pos + sizeof(cap));
- if (ret != sizeof(flags)) {
- PMD_INIT_LOG(DEBUG,
- "failed to read pci cap at pos:"
- " %x ret %d", pos + 2, ret);
- break;
- }
-
- if (flags & PCI_MSIX_ENABLE)
- return VIRTIO_MSIX_ENABLED;
- else
- return VIRTIO_MSIX_DISABLED;
- }
-
- pos = cap[1];
- }
-
- return VIRTIO_MSIX_NONE;
-}
-
void vtpci_legacy_ioport_unmap(struct virtio_hw *hw)
{
rte_pci_ioport_unmap(VTPCI_IO(hw));
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 19d56c920c..4ee890ffda 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -42,13 +42,6 @@ struct virtnet_ctl;
#define VIRTIO_MSI_QUEUE_VECTOR 22 /* vector for selected VQ notifications
(16, RW) */
-/* The bit of the ISR which indicates a device has an interrupt. */
-#define VIRTIO_PCI_ISR_INTR 0x1
-/* The bit of the ISR which indicates a device configuration change. */
-#define VIRTIO_PCI_ISR_CONFIG 0x2
-/* Vector value used to disable MSI for queue. */
-#define VIRTIO_MSI_NO_VECTOR 0xFFFF
-
/* Common configuration */
#define VIRTIO_PCI_CAP_COMMON_CFG 1
/* Notifications */
@@ -103,12 +96,18 @@ struct virtio_pci_common_cfg {
uint32_t queue_used_hi; /* read-write */
};
+enum virtio_msix_status {
+ VIRTIO_MSIX_NONE = 0,
+ VIRTIO_MSIX_DISABLED = 1,
+ VIRTIO_MSIX_ENABLED = 2
+};
struct virtio_pci_dev {
struct virtio_hw hw;
struct rte_pci_device *pci_dev;
struct virtio_pci_common_cfg *common_cfg;
struct virtio_net_config *dev_cfg;
+ enum virtio_msix_status msix_status;
uint8_t *isr;
uint16_t *notify_base;
uint32_t notify_off_multiplier;
@@ -126,22 +125,10 @@ struct virtio_pci_dev {
/* The alignment to use between consumer and producer parts of vring. */
#define VIRTIO_PCI_VRING_ALIGN 4096
-enum virtio_msix_status {
- VIRTIO_MSIX_NONE = 0,
- VIRTIO_MSIX_DISABLED = 1,
- VIRTIO_MSIX_ENABLED = 2
-};
-
-
/*
* Function declaration from virtio_pci.c
*/
int vtpci_init(struct rte_pci_device *pci_dev, struct virtio_pci_dev *dev);
-
-uint8_t vtpci_isr(struct virtio_hw *);
-
-enum virtio_msix_status vtpci_msix_detect(struct rte_pci_device *dev);
-
void vtpci_legacy_ioport_unmap(struct virtio_hw *hw);
int vtpci_legacy_ioport_map(struct virtio_hw *hw);
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 0f252c0732..eacb268297 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -339,7 +339,7 @@ virtio_user_get_isr(struct virtio_hw *hw __rte_unused)
/* rxq interrupts and config interrupt are separated in virtio-user,
* here we only report config change.
*/
- return VIRTIO_PCI_ISR_CONFIG;
+ return VIRTIO_ISR_CONFIG;
}
static uint16_t
@@ -640,7 +640,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
* MSIX is required to enable LSC (see virtio_init_device).
* Here just pretend that we support msix.
*/
- hw->use_msix = 1;
+ hw->intr_lsc = 1;
hw->use_vec_rx = 0;
hw->use_vec_tx = 0;
hw->use_inorder_rx = 0;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic
2020-12-20 21:13 ` [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic Maxime Coquelin
@ 2020-12-30 3:17 ` Xia, Chenbo
2021-01-14 8:43 ` Maxime Coquelin
2021-01-06 16:07 ` David Marchand
1 sibling, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:17 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 20/40] net/virtio: make interrupt handling more generic
>
> This patch aims at isolating MSIX notion into PCI
> layer.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.c | 6 ++
> drivers/net/virtio/virtio.h | 11 +-
> drivers/net/virtio/virtio_ethdev.c | 7 +-
> drivers/net/virtio/virtio_pci.c | 131 ++++++++++++------------
> drivers/net/virtio/virtio_pci.h | 25 ++---
> drivers/net/virtio/virtio_user_ethdev.c | 4 +-
> 6 files changed, 90 insertions(+), 94 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
> index ba3203e68b..7e1e77797f 100644
> --- a/drivers/net/virtio/virtio.c
> +++ b/drivers/net/virtio/virtio.c
> @@ -63,3 +63,9 @@ virtio_get_status(struct virtio_hw *hw)
> {
> return VIRTIO_OPS(hw)->get_status(hw);
> }
[snip]
>
> static uint16_t
> @@ -640,7 +640,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
> * MSIX is required to enable LSC (see virtio_init_device).
> * Here just pretend that we support msix.
> */
> - hw->use_msix = 1;
> + hw->intr_lsc = 1;
As virtio-user does not have the notion of msi-x, should we also clean up the code
comments? I mean 'MSIX is required ... that we support msix'.
Thanks!
Chenbo
> hw->use_vec_rx = 0;
> hw->use_vec_tx = 0;
> hw->use_inorder_rx = 0;
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic
2020-12-30 3:17 ` Xia, Chenbo
@ 2021-01-14 8:43 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 8:43 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 12/30/20 4:17 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 20/40] net/virtio: make interrupt handling more generic
>>
>> This patch aims at isolating MSIX notion into PCI
>> layer.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio.c | 6 ++
>> drivers/net/virtio/virtio.h | 11 +-
>> drivers/net/virtio/virtio_ethdev.c | 7 +-
>> drivers/net/virtio/virtio_pci.c | 131 ++++++++++++------------
>> drivers/net/virtio/virtio_pci.h | 25 ++---
>> drivers/net/virtio/virtio_user_ethdev.c | 4 +-
>> 6 files changed, 90 insertions(+), 94 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio.c b/drivers/net/virtio/virtio.c
>> index ba3203e68b..7e1e77797f 100644
>> --- a/drivers/net/virtio/virtio.c
>> +++ b/drivers/net/virtio/virtio.c
>> @@ -63,3 +63,9 @@ virtio_get_status(struct virtio_hw *hw)
>> {
>> return VIRTIO_OPS(hw)->get_status(hw);
>> }
>
> [snip]
>
>>
>> static uint16_t
>> @@ -640,7 +640,7 @@ virtio_user_eth_dev_alloc(struct rte_vdev_device *vdev)
>> * MSIX is required to enable LSC (see virtio_init_device).
>> * Here just pretend that we support msix.
>> */
>> - hw->use_msix = 1;
>> + hw->intr_lsc = 1;
>
> As virtio-user does not have the notion of msi-x, should we also clean up the code
> comments? I mean 'MSIX is required ... that we support msix'.
The comment no more makes sense, I'll just drop it.
Thanks,
Maxime
> Thanks!
> Chenbo
>
>> hw->use_vec_rx = 0;
>> hw->use_vec_tx = 0;
>> hw->use_inorder_rx = 0;
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic
2020-12-20 21:13 ` [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic Maxime Coquelin
2020-12-30 3:17 ` Xia, Chenbo
@ 2021-01-06 16:07 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 16:07 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch aims at isolating MSIX notion into PCI
> layer.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (19 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 20/40] net/virtio: make interrupt handling more generic Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:18 ` Xia, Chenbo
2021-01-06 16:13 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code Maxime Coquelin
` (19 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves vring alignment define to the generic
Virtio header.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio.h | 3 +++
drivers/net/virtio/virtio_ethdev.c | 10 +++++-----
drivers/net/virtio/virtio_pci.c | 2 +-
drivers/net/virtio/virtio_pci.h | 3 ---
drivers/net/virtio/virtio_user_ethdev.c | 4 ++--
5 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
index f44125f48a..964e15bb22 100644
--- a/drivers/net/virtio/virtio.h
+++ b/drivers/net/virtio/virtio.h
@@ -131,6 +131,9 @@
/* Vector value used to disable MSI for queue. */
#define VIRTIO_MSI_NO_VECTOR 0xFFFF
+/* The alignment to use between consumer and producer parts of vring. */
+#define VIRTIO_VRING_ALIGN 4096
+
/*
* This structure is just a reference to read
* net device specific config space; it just a chodu structure
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 99a2dd24c4..1ca8715832 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -407,12 +407,12 @@ virtio_init_vring(struct virtqueue *vq)
memset(vq->vq_descx, 0, sizeof(struct vq_desc_extra) * vq->vq_nentries);
if (virtio_with_packed_queue(vq->hw)) {
vring_init_packed(&vq->vq_packed.ring, ring_mem,
- VIRTIO_PCI_VRING_ALIGN, size);
+ VIRTIO_VRING_ALIGN, size);
vring_desc_init_packed(vq, size);
} else {
struct vring *vr = &vq->vq_split.ring;
- vring_init_split(vr, ring_mem, VIRTIO_PCI_VRING_ALIGN, size);
+ vring_init_split(vr, ring_mem, VIRTIO_VRING_ALIGN, size);
vring_desc_init_split(vr->desc, size);
}
/*
@@ -497,14 +497,14 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
/*
* Reserve a memzone for vring elements
*/
- size = vring_size(hw, vq_size, VIRTIO_PCI_VRING_ALIGN);
- vq->vq_ring_size = RTE_ALIGN_CEIL(size, VIRTIO_PCI_VRING_ALIGN);
+ size = vring_size(hw, vq_size, VIRTIO_VRING_ALIGN);
+ vq->vq_ring_size = RTE_ALIGN_CEIL(size, VIRTIO_VRING_ALIGN);
PMD_INIT_LOG(DEBUG, "vring_size: %d, rounded_vring_size: %d",
size, vq->vq_ring_size);
mz = rte_memzone_reserve_aligned(vq_name, vq->vq_ring_size,
numa_node, RTE_MEMZONE_IOVA_CONTIG,
- VIRTIO_PCI_VRING_ALIGN);
+ VIRTIO_VRING_ALIGN);
if (mz == NULL) {
if (rte_errno == EEXIST)
mz = rte_memzone_lookup(vq_name);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 01a437a1b4..f8532c5684 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -432,7 +432,7 @@ modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq)
avail_addr = desc_addr + vq->vq_nentries * sizeof(struct vring_desc);
used_addr = RTE_ALIGN_CEIL(avail_addr + offsetof(struct vring_avail,
ring[vq->vq_nentries]),
- VIRTIO_PCI_VRING_ALIGN);
+ VIRTIO_VRING_ALIGN);
rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
index 4ee890ffda..2cb3ebf93d 100644
--- a/drivers/net/virtio/virtio_pci.h
+++ b/drivers/net/virtio/virtio_pci.h
@@ -122,9 +122,6 @@ struct virtio_pci_dev {
*/
#define VIRTIO_PCI_QUEUE_ADDR_SHIFT 12
-/* The alignment to use between consumer and producer parts of vring. */
-#define VIRTIO_PCI_VRING_ALIGN 4096
-
/*
* Function declaration from virtio_pci.c
*/
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index eacb268297..283f5c7a36 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -388,7 +388,7 @@ virtio_user_setup_queue_packed(struct virtqueue *vq,
sizeof(struct vring_packed_desc);
used_addr = RTE_ALIGN_CEIL(avail_addr +
sizeof(struct vring_packed_desc_event),
- VIRTIO_PCI_VRING_ALIGN);
+ VIRTIO_VRING_ALIGN);
vring->num = vq->vq_nentries;
vring->desc = (void *)(uintptr_t)desc_addr;
vring->driver = (void *)(uintptr_t)avail_addr;
@@ -410,7 +410,7 @@ virtio_user_setup_queue_split(struct virtqueue *vq, struct virtio_user_dev *dev)
avail_addr = desc_addr + vq->vq_nentries * sizeof(struct vring_desc);
used_addr = RTE_ALIGN_CEIL(avail_addr + offsetof(struct vring_avail,
ring[vq->vq_nentries]),
- VIRTIO_PCI_VRING_ALIGN);
+ VIRTIO_VRING_ALIGN);
dev->vrings[queue_idx].num = vq->vq_nentries;
dev->vrings[queue_idx].desc = (void *)(uintptr_t)desc_addr;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header Maxime Coquelin
@ 2020-12-30 3:18 ` Xia, Chenbo
2021-01-06 16:13 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:18 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 21/40] net/virtio: move vring alignment to generic header
>
> This patch moves vring alignment define to the generic
> Virtio header.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio.h | 3 +++
> drivers/net/virtio/virtio_ethdev.c | 10 +++++-----
> drivers/net/virtio/virtio_pci.c | 2 +-
> drivers/net/virtio/virtio_pci.h | 3 ---
> drivers/net/virtio/virtio_user_ethdev.c | 4 ++--
> 5 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio.h b/drivers/net/virtio/virtio.h
> index f44125f48a..964e15bb22 100644
> --- a/drivers/net/virtio/virtio.h
> +++ b/drivers/net/virtio/virtio.h
> @@ -131,6 +131,9 @@
> /* Vector value used to disable MSI for queue. */
> #define VIRTIO_MSI_NO_VECTOR 0xFFFF
>
> +/* The alignment to use between consumer and producer parts of vring. */
> +#define VIRTIO_VRING_ALIGN 4096
> +
> /*
> * This structure is just a reference to read
> * net device specific config space; it just a chodu structure
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 99a2dd24c4..1ca8715832 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -407,12 +407,12 @@ virtio_init_vring(struct virtqueue *vq)
> memset(vq->vq_descx, 0, sizeof(struct vq_desc_extra) * vq->vq_nentries);
> if (virtio_with_packed_queue(vq->hw)) {
> vring_init_packed(&vq->vq_packed.ring, ring_mem,
> - VIRTIO_PCI_VRING_ALIGN, size);
> + VIRTIO_VRING_ALIGN, size);
> vring_desc_init_packed(vq, size);
> } else {
> struct vring *vr = &vq->vq_split.ring;
>
> - vring_init_split(vr, ring_mem, VIRTIO_PCI_VRING_ALIGN, size);
> + vring_init_split(vr, ring_mem, VIRTIO_VRING_ALIGN, size);
> vring_desc_init_split(vr->desc, size);
> }
> /*
> @@ -497,14 +497,14 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> /*
> * Reserve a memzone for vring elements
> */
> - size = vring_size(hw, vq_size, VIRTIO_PCI_VRING_ALIGN);
> - vq->vq_ring_size = RTE_ALIGN_CEIL(size, VIRTIO_PCI_VRING_ALIGN);
> + size = vring_size(hw, vq_size, VIRTIO_VRING_ALIGN);
> + vq->vq_ring_size = RTE_ALIGN_CEIL(size, VIRTIO_VRING_ALIGN);
> PMD_INIT_LOG(DEBUG, "vring_size: %d, rounded_vring_size: %d",
> size, vq->vq_ring_size);
>
> mz = rte_memzone_reserve_aligned(vq_name, vq->vq_ring_size,
> numa_node, RTE_MEMZONE_IOVA_CONTIG,
> - VIRTIO_PCI_VRING_ALIGN);
> + VIRTIO_VRING_ALIGN);
> if (mz == NULL) {
> if (rte_errno == EEXIST)
> mz = rte_memzone_lookup(vq_name);
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 01a437a1b4..f8532c5684 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -432,7 +432,7 @@ modern_setup_queue(struct virtio_hw *hw, struct virtqueue
> *vq)
> avail_addr = desc_addr + vq->vq_nentries * sizeof(struct vring_desc);
> used_addr = RTE_ALIGN_CEIL(avail_addr + offsetof(struct vring_avail,
> ring[vq->vq_nentries]),
> - VIRTIO_PCI_VRING_ALIGN);
> + VIRTIO_VRING_ALIGN);
>
> rte_write16(vq->vq_queue_index, &dev->common_cfg->queue_select);
>
> diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h
> index 4ee890ffda..2cb3ebf93d 100644
> --- a/drivers/net/virtio/virtio_pci.h
> +++ b/drivers/net/virtio/virtio_pci.h
> @@ -122,9 +122,6 @@ struct virtio_pci_dev {
> */
> #define VIRTIO_PCI_QUEUE_ADDR_SHIFT 12
>
> -/* The alignment to use between consumer and producer parts of vring. */
> -#define VIRTIO_PCI_VRING_ALIGN 4096
> -
> /*
> * Function declaration from virtio_pci.c
> */
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index eacb268297..283f5c7a36 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -388,7 +388,7 @@ virtio_user_setup_queue_packed(struct virtqueue *vq,
> sizeof(struct vring_packed_desc);
> used_addr = RTE_ALIGN_CEIL(avail_addr +
> sizeof(struct vring_packed_desc_event),
> - VIRTIO_PCI_VRING_ALIGN);
> + VIRTIO_VRING_ALIGN);
> vring->num = vq->vq_nentries;
> vring->desc = (void *)(uintptr_t)desc_addr;
> vring->driver = (void *)(uintptr_t)avail_addr;
> @@ -410,7 +410,7 @@ virtio_user_setup_queue_split(struct virtqueue *vq, struct
> virtio_user_dev *dev)
> avail_addr = desc_addr + vq->vq_nentries * sizeof(struct vring_desc);
> used_addr = RTE_ALIGN_CEIL(avail_addr + offsetof(struct vring_avail,
> ring[vq->vq_nentries]),
> - VIRTIO_PCI_VRING_ALIGN);
> + VIRTIO_VRING_ALIGN);
>
> dev->vrings[queue_idx].num = vq->vq_nentries;
> dev->vrings[queue_idx].desc = (void *)(uintptr_t)desc_addr;
> --
> 2.29.2
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header
2020-12-20 21:13 ` [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header Maxime Coquelin
2020-12-30 3:18 ` Xia, Chenbo
@ 2021-01-06 16:13 ` David Marchand
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-06 16:13 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch moves vring alignment define to the generic
> Virtio header.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (20 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 21/40] net/virtio: move vring alignment to generic header Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-30 3:25 ` Xia, Chenbo
2021-01-06 16:18 ` David Marchand
2020-12-20 21:13 ` [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent Maxime Coquelin
` (18 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch finalizes the bus isolation part of this
refactoring.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_ethdev.c | 21 +++++++++------------
drivers/net/virtio/virtio_rxtx.c | 18 +++++++++---------
drivers/net/virtio/virtio_rxtx_packed_avx.c | 2 +-
drivers/net/virtio/virtio_user/vhost.h | 4 +++-
drivers/net/virtio/virtqueue.c | 2 +-
drivers/net/virtio/virtqueue.h | 6 +++---
6 files changed, 26 insertions(+), 27 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 1ca8715832..96871b7b70 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -9,14 +9,11 @@
#include <unistd.h>
#include <rte_ethdev_driver.h>
-#include <rte_ethdev_pci.h>
#include <rte_memcpy.h>
#include <rte_string_fns.h>
#include <rte_memzone.h>
#include <rte_malloc.h>
#include <rte_branch_prediction.h>
-#include <rte_pci.h>
-#include <rte_bus_pci.h>
#include <rte_ether.h>
#include <rte_ip.h>
#include <rte_arp.h>
@@ -32,7 +29,7 @@
#include <rte_kvargs.h>
#include "virtio_ethdev.h"
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtio_logs.h"
#include "virtqueue.h"
#include "virtio_rxtx.h"
@@ -422,7 +419,7 @@ virtio_init_vring(struct virtqueue *vq)
}
static int
-virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
+virtio_init_queue(struct rte_eth_dev *dev, uint16_t queue_idx)
{
char vq_name[VIRTQUEUE_MAX_NAME_SZ];
char vq_hdr_name[VIRTQUEUE_MAX_NAME_SZ];
@@ -435,18 +432,18 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
struct virtqueue *vq;
size_t sz_hdr_mz = 0;
void *sw_ring = NULL;
- int queue_type = virtio_get_queue_type(hw, vtpci_queue_idx);
+ int queue_type = virtio_get_queue_type(hw, queue_idx);
int ret;
int numa_node = dev->device->numa_node;
PMD_INIT_LOG(INFO, "setting up queue: %u on NUMA node %d",
- vtpci_queue_idx, numa_node);
+ queue_idx, numa_node);
/*
* Read the virtqueue size from the Queue Size field
* Always power of 2 and if 0 virtqueue does not exist
*/
- vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
+ vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, queue_idx);
PMD_INIT_LOG(DEBUG, "vq_size: %u", vq_size);
if (vq_size == 0) {
PMD_INIT_LOG(ERR, "virtqueue does not exist");
@@ -459,7 +456,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
}
snprintf(vq_name, sizeof(vq_name), "port%d_vq%d",
- dev->data->port_id, vtpci_queue_idx);
+ dev->data->port_id, queue_idx);
size = RTE_ALIGN_CEIL(sizeof(*vq) +
vq_size * sizeof(struct vq_desc_extra),
@@ -481,10 +478,10 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
PMD_INIT_LOG(ERR, "can not allocate vq");
return -ENOMEM;
}
- hw->vqs[vtpci_queue_idx] = vq;
+ hw->vqs[queue_idx] = vq;
vq->hw = hw;
- vq->vq_queue_index = vtpci_queue_idx;
+ vq->vq_queue_index = queue_idx;
vq->vq_nentries = vq_size;
if (virtio_with_packed_queue(hw)) {
vq->vq_packed.used_wrap_counter = 1;
@@ -527,7 +524,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
if (sz_hdr_mz) {
snprintf(vq_hdr_name, sizeof(vq_hdr_name), "port%d_vq%d_hdr",
- dev->data->port_id, vtpci_queue_idx);
+ dev->data->port_id, queue_idx);
hdr_mz = rte_memzone_reserve_aligned(vq_hdr_name, sz_hdr_mz,
numa_node, RTE_MEMZONE_IOVA_CONTIG,
RTE_CACHE_LINE_SIZE);
diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index 10989118b0..aca6eb9cd0 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -27,7 +27,7 @@
#include "virtio_logs.h"
#include "virtio_ethdev.h"
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtqueue.h"
#include "virtio_rxtx.h"
#include "virtio_rxtx_simple.h"
@@ -628,9 +628,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp)
{
- uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
+ uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
struct virtio_hw *hw = dev->data->dev_private;
- struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
+ struct virtqueue *vq = hw->vqs[vq_idx];
struct virtnet_rx *rxvq;
uint16_t rx_free_thresh;
@@ -678,9 +678,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
int
virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
{
- uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
+ uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
struct virtio_hw *hw = dev->data->dev_private;
- struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
+ struct virtqueue *vq = hw->vqs[vq_idx];
struct virtnet_rx *rxvq = &vq->rxq;
struct rte_mbuf *m;
uint16_t desc_idx;
@@ -779,9 +779,9 @@ virtio_dev_tx_queue_setup(struct rte_eth_dev *dev,
unsigned int socket_id __rte_unused,
const struct rte_eth_txconf *tx_conf)
{
- uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
+ uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
struct virtio_hw *hw = dev->data->dev_private;
- struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
+ struct virtqueue *vq = hw->vqs[vq_idx];
struct virtnet_tx *txvq;
uint16_t tx_free_thresh;
@@ -823,9 +823,9 @@ int
virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
uint16_t queue_idx)
{
- uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
+ uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
struct virtio_hw *hw = dev->data->dev_private;
- struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
+ struct virtqueue *vq = hw->vqs[vq_idx];
PMD_INIT_FUNC_TRACE();
diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c b/drivers/net/virtio/virtio_rxtx_packed_avx.c
index c272766a9f..a3dcc01a43 100644
--- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
+++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
@@ -12,7 +12,7 @@
#include "virtio_logs.h"
#include "virtio_ethdev.h"
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtqueue.h"
#define BYTE_SIZE 8
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 210a3704e7..9d2a8505b3 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -9,7 +9,9 @@
#include <linux/types.h>
#include <linux/ioctl.h>
-#include "../virtio_pci.h"
+#include <rte_errno.h>
+
+#include "../virtio.h"
#include "../virtio_logs.h"
#include "../virtqueue.h"
diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
index 59a2cb6599..1f9af3c31b 100644
--- a/drivers/net/virtio/virtqueue.c
+++ b/drivers/net/virtio/virtqueue.c
@@ -7,7 +7,7 @@
#include "virtqueue.h"
#include "virtio_logs.h"
-#include "virtio_pci.h"
+#include "virtio.h"
#include "virtio_rxtx_simple.h"
/*
diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
index 6c1df6f8e5..9274c48080 100644
--- a/drivers/net/virtio/virtqueue.h
+++ b/drivers/net/virtio/virtqueue.h
@@ -449,11 +449,11 @@ virtqueue_full(const struct virtqueue *vq)
}
static inline int
-virtio_get_queue_type(struct virtio_hw *hw, uint16_t vtpci_queue_idx)
+virtio_get_queue_type(struct virtio_hw *hw, uint16_t vq_idx)
{
- if (vtpci_queue_idx == hw->max_queue_pairs * 2)
+ if (vq_idx == hw->max_queue_pairs * 2)
return VTNET_CQ;
- else if (vtpci_queue_idx % 2 == 0)
+ else if (vq_idx % 2 == 0)
return VTNET_RQ;
else
return VTNET_TQ;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
2020-12-20 21:13 ` [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code Maxime Coquelin
@ 2020-12-30 3:25 ` Xia, Chenbo
2021-01-14 8:46 ` Maxime Coquelin
2021-01-06 16:18 ` David Marchand
1 sibling, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2020-12-30 3:25 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
>
> This patch finalizes the bus isolation part of this
> refactoring.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
I think this one is also the remove-dependency part? Because you said first 21 patches
are the first part :P
I think the first part is very great clean-up. It does make our code cleaner!
For this patch:
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
> ---
> drivers/net/virtio/virtio_ethdev.c | 21 +++++++++------------
> drivers/net/virtio/virtio_rxtx.c | 18 +++++++++---------
> drivers/net/virtio/virtio_rxtx_packed_avx.c | 2 +-
> drivers/net/virtio/virtio_user/vhost.h | 4 +++-
> drivers/net/virtio/virtqueue.c | 2 +-
> drivers/net/virtio/virtqueue.h | 6 +++---
> 6 files changed, 26 insertions(+), 27 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 1ca8715832..96871b7b70 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -9,14 +9,11 @@
> #include <unistd.h>
>
> #include <rte_ethdev_driver.h>
> -#include <rte_ethdev_pci.h>
> #include <rte_memcpy.h>
> #include <rte_string_fns.h>
> #include <rte_memzone.h>
> #include <rte_malloc.h>
> #include <rte_branch_prediction.h>
> -#include <rte_pci.h>
> -#include <rte_bus_pci.h>
> #include <rte_ether.h>
> #include <rte_ip.h>
> #include <rte_arp.h>
> @@ -32,7 +29,7 @@
> #include <rte_kvargs.h>
>
> #include "virtio_ethdev.h"
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtio_logs.h"
> #include "virtqueue.h"
> #include "virtio_rxtx.h"
> @@ -422,7 +419,7 @@ virtio_init_vring(struct virtqueue *vq)
> }
>
> static int
> -virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
> +virtio_init_queue(struct rte_eth_dev *dev, uint16_t queue_idx)
> {
> char vq_name[VIRTQUEUE_MAX_NAME_SZ];
> char vq_hdr_name[VIRTQUEUE_MAX_NAME_SZ];
> @@ -435,18 +432,18 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> struct virtqueue *vq;
> size_t sz_hdr_mz = 0;
> void *sw_ring = NULL;
> - int queue_type = virtio_get_queue_type(hw, vtpci_queue_idx);
> + int queue_type = virtio_get_queue_type(hw, queue_idx);
> int ret;
> int numa_node = dev->device->numa_node;
>
> PMD_INIT_LOG(INFO, "setting up queue: %u on NUMA node %d",
> - vtpci_queue_idx, numa_node);
> + queue_idx, numa_node);
>
> /*
> * Read the virtqueue size from the Queue Size field
> * Always power of 2 and if 0 virtqueue does not exist
> */
> - vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
> + vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, queue_idx);
> PMD_INIT_LOG(DEBUG, "vq_size: %u", vq_size);
> if (vq_size == 0) {
> PMD_INIT_LOG(ERR, "virtqueue does not exist");
> @@ -459,7 +456,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> }
>
> snprintf(vq_name, sizeof(vq_name), "port%d_vq%d",
> - dev->data->port_id, vtpci_queue_idx);
> + dev->data->port_id, queue_idx);
>
> size = RTE_ALIGN_CEIL(sizeof(*vq) +
> vq_size * sizeof(struct vq_desc_extra),
> @@ -481,10 +478,10 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
> PMD_INIT_LOG(ERR, "can not allocate vq");
> return -ENOMEM;
> }
> - hw->vqs[vtpci_queue_idx] = vq;
> + hw->vqs[queue_idx] = vq;
>
> vq->hw = hw;
> - vq->vq_queue_index = vtpci_queue_idx;
> + vq->vq_queue_index = queue_idx;
> vq->vq_nentries = vq_size;
> if (virtio_with_packed_queue(hw)) {
> vq->vq_packed.used_wrap_counter = 1;
> @@ -527,7 +524,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
> vtpci_queue_idx)
>
> if (sz_hdr_mz) {
> snprintf(vq_hdr_name, sizeof(vq_hdr_name), "port%d_vq%d_hdr",
> - dev->data->port_id, vtpci_queue_idx);
> + dev->data->port_id, queue_idx);
> hdr_mz = rte_memzone_reserve_aligned(vq_hdr_name, sz_hdr_mz,
> numa_node, RTE_MEMZONE_IOVA_CONTIG,
> RTE_CACHE_LINE_SIZE);
> diff --git a/drivers/net/virtio/virtio_rxtx.c
> b/drivers/net/virtio/virtio_rxtx.c
> index 10989118b0..aca6eb9cd0 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -27,7 +27,7 @@
>
> #include "virtio_logs.h"
> #include "virtio_ethdev.h"
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtqueue.h"
> #include "virtio_rxtx.h"
> #include "virtio_rxtx_simple.h"
> @@ -628,9 +628,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
> const struct rte_eth_rxconf *rx_conf,
> struct rte_mempool *mp)
> {
> - uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
> + uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
> struct virtio_hw *hw = dev->data->dev_private;
> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
> + struct virtqueue *vq = hw->vqs[vq_idx];
> struct virtnet_rx *rxvq;
> uint16_t rx_free_thresh;
>
> @@ -678,9 +678,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
> int
> virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
> {
> - uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
> + uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
> struct virtio_hw *hw = dev->data->dev_private;
> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
> + struct virtqueue *vq = hw->vqs[vq_idx];
> struct virtnet_rx *rxvq = &vq->rxq;
> struct rte_mbuf *m;
> uint16_t desc_idx;
> @@ -779,9 +779,9 @@ virtio_dev_tx_queue_setup(struct rte_eth_dev *dev,
> unsigned int socket_id __rte_unused,
> const struct rte_eth_txconf *tx_conf)
> {
> - uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
> + uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
> struct virtio_hw *hw = dev->data->dev_private;
> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
> + struct virtqueue *vq = hw->vqs[vq_idx];
> struct virtnet_tx *txvq;
> uint16_t tx_free_thresh;
>
> @@ -823,9 +823,9 @@ int
> virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
> uint16_t queue_idx)
> {
> - uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
> + uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
> struct virtio_hw *hw = dev->data->dev_private;
> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
> + struct virtqueue *vq = hw->vqs[vq_idx];
>
> PMD_INIT_FUNC_TRACE();
>
> diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> index c272766a9f..a3dcc01a43 100644
> --- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
> +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
> @@ -12,7 +12,7 @@
>
> #include "virtio_logs.h"
> #include "virtio_ethdev.h"
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtqueue.h"
>
> #define BYTE_SIZE 8
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index 210a3704e7..9d2a8505b3 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -9,7 +9,9 @@
> #include <linux/types.h>
> #include <linux/ioctl.h>
>
> -#include "../virtio_pci.h"
> +#include <rte_errno.h>
> +
> +#include "../virtio.h"
> #include "../virtio_logs.h"
> #include "../virtqueue.h"
>
> diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
> index 59a2cb6599..1f9af3c31b 100644
> --- a/drivers/net/virtio/virtqueue.c
> +++ b/drivers/net/virtio/virtqueue.c
> @@ -7,7 +7,7 @@
>
> #include "virtqueue.h"
> #include "virtio_logs.h"
> -#include "virtio_pci.h"
> +#include "virtio.h"
> #include "virtio_rxtx_simple.h"
>
> /*
> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
> index 6c1df6f8e5..9274c48080 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -449,11 +449,11 @@ virtqueue_full(const struct virtqueue *vq)
> }
>
> static inline int
> -virtio_get_queue_type(struct virtio_hw *hw, uint16_t vtpci_queue_idx)
> +virtio_get_queue_type(struct virtio_hw *hw, uint16_t vq_idx)
> {
> - if (vtpci_queue_idx == hw->max_queue_pairs * 2)
> + if (vq_idx == hw->max_queue_pairs * 2)
> return VTNET_CQ;
> - else if (vtpci_queue_idx % 2 == 0)
> + else if (vq_idx % 2 == 0)
> return VTNET_RQ;
> else
> return VTNET_TQ;
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
2020-12-30 3:25 ` Xia, Chenbo
@ 2021-01-14 8:46 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-14 8:46 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 12/30/20 4:25 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
>>
>> This patch finalizes the bus isolation part of this
>> refactoring.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
> I think this one is also the remove-dependency part? Because you said first 21 patches
> are the first part :P
Yes, that's what I mean by bus isolation.
> I think the first part is very great clean-up. It does make our code cleaner!
>
> For this patch:
>
> Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Thanks, your review is much apopreciated.
Maxime
>
>> ---
>> drivers/net/virtio/virtio_ethdev.c | 21 +++++++++------------
>> drivers/net/virtio/virtio_rxtx.c | 18 +++++++++---------
>> drivers/net/virtio/virtio_rxtx_packed_avx.c | 2 +-
>> drivers/net/virtio/virtio_user/vhost.h | 4 +++-
>> drivers/net/virtio/virtqueue.c | 2 +-
>> drivers/net/virtio/virtqueue.h | 6 +++---
>> 6 files changed, 26 insertions(+), 27 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_ethdev.c
>> b/drivers/net/virtio/virtio_ethdev.c
>> index 1ca8715832..96871b7b70 100644
>> --- a/drivers/net/virtio/virtio_ethdev.c
>> +++ b/drivers/net/virtio/virtio_ethdev.c
>> @@ -9,14 +9,11 @@
>> #include <unistd.h>
>>
>> #include <rte_ethdev_driver.h>
>> -#include <rte_ethdev_pci.h>
>> #include <rte_memcpy.h>
>> #include <rte_string_fns.h>
>> #include <rte_memzone.h>
>> #include <rte_malloc.h>
>> #include <rte_branch_prediction.h>
>> -#include <rte_pci.h>
>> -#include <rte_bus_pci.h>
>> #include <rte_ether.h>
>> #include <rte_ip.h>
>> #include <rte_arp.h>
>> @@ -32,7 +29,7 @@
>> #include <rte_kvargs.h>
>>
>> #include "virtio_ethdev.h"
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> #include "virtio_logs.h"
>> #include "virtqueue.h"
>> #include "virtio_rxtx.h"
>> @@ -422,7 +419,7 @@ virtio_init_vring(struct virtqueue *vq)
>> }
>>
>> static int
>> -virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx)
>> +virtio_init_queue(struct rte_eth_dev *dev, uint16_t queue_idx)
>> {
>> char vq_name[VIRTQUEUE_MAX_NAME_SZ];
>> char vq_hdr_name[VIRTQUEUE_MAX_NAME_SZ];
>> @@ -435,18 +432,18 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
>> vtpci_queue_idx)
>> struct virtqueue *vq;
>> size_t sz_hdr_mz = 0;
>> void *sw_ring = NULL;
>> - int queue_type = virtio_get_queue_type(hw, vtpci_queue_idx);
>> + int queue_type = virtio_get_queue_type(hw, queue_idx);
>> int ret;
>> int numa_node = dev->device->numa_node;
>>
>> PMD_INIT_LOG(INFO, "setting up queue: %u on NUMA node %d",
>> - vtpci_queue_idx, numa_node);
>> + queue_idx, numa_node);
>>
>> /*
>> * Read the virtqueue size from the Queue Size field
>> * Always power of 2 and if 0 virtqueue does not exist
>> */
>> - vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, vtpci_queue_idx);
>> + vq_size = VIRTIO_OPS(hw)->get_queue_num(hw, queue_idx);
>> PMD_INIT_LOG(DEBUG, "vq_size: %u", vq_size);
>> if (vq_size == 0) {
>> PMD_INIT_LOG(ERR, "virtqueue does not exist");
>> @@ -459,7 +456,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
>> vtpci_queue_idx)
>> }
>>
>> snprintf(vq_name, sizeof(vq_name), "port%d_vq%d",
>> - dev->data->port_id, vtpci_queue_idx);
>> + dev->data->port_id, queue_idx);
>>
>> size = RTE_ALIGN_CEIL(sizeof(*vq) +
>> vq_size * sizeof(struct vq_desc_extra),
>> @@ -481,10 +478,10 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
>> vtpci_queue_idx)
>> PMD_INIT_LOG(ERR, "can not allocate vq");
>> return -ENOMEM;
>> }
>> - hw->vqs[vtpci_queue_idx] = vq;
>> + hw->vqs[queue_idx] = vq;
>>
>> vq->hw = hw;
>> - vq->vq_queue_index = vtpci_queue_idx;
>> + vq->vq_queue_index = queue_idx;
>> vq->vq_nentries = vq_size;
>> if (virtio_with_packed_queue(hw)) {
>> vq->vq_packed.used_wrap_counter = 1;
>> @@ -527,7 +524,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t
>> vtpci_queue_idx)
>>
>> if (sz_hdr_mz) {
>> snprintf(vq_hdr_name, sizeof(vq_hdr_name), "port%d_vq%d_hdr",
>> - dev->data->port_id, vtpci_queue_idx);
>> + dev->data->port_id, queue_idx);
>> hdr_mz = rte_memzone_reserve_aligned(vq_hdr_name, sz_hdr_mz,
>> numa_node, RTE_MEMZONE_IOVA_CONTIG,
>> RTE_CACHE_LINE_SIZE);
>> diff --git a/drivers/net/virtio/virtio_rxtx.c
>> b/drivers/net/virtio/virtio_rxtx.c
>> index 10989118b0..aca6eb9cd0 100644
>> --- a/drivers/net/virtio/virtio_rxtx.c
>> +++ b/drivers/net/virtio/virtio_rxtx.c
>> @@ -27,7 +27,7 @@
>>
>> #include "virtio_logs.h"
>> #include "virtio_ethdev.h"
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> #include "virtqueue.h"
>> #include "virtio_rxtx.h"
>> #include "virtio_rxtx_simple.h"
>> @@ -628,9 +628,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
>> const struct rte_eth_rxconf *rx_conf,
>> struct rte_mempool *mp)
>> {
>> - uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
>> + uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
>> struct virtio_hw *hw = dev->data->dev_private;
>> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
>> + struct virtqueue *vq = hw->vqs[vq_idx];
>> struct virtnet_rx *rxvq;
>> uint16_t rx_free_thresh;
>>
>> @@ -678,9 +678,9 @@ virtio_dev_rx_queue_setup(struct rte_eth_dev *dev,
>> int
>> virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, uint16_t queue_idx)
>> {
>> - uint16_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
>> + uint16_t vq_idx = 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX;
>> struct virtio_hw *hw = dev->data->dev_private;
>> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
>> + struct virtqueue *vq = hw->vqs[vq_idx];
>> struct virtnet_rx *rxvq = &vq->rxq;
>> struct rte_mbuf *m;
>> uint16_t desc_idx;
>> @@ -779,9 +779,9 @@ virtio_dev_tx_queue_setup(struct rte_eth_dev *dev,
>> unsigned int socket_id __rte_unused,
>> const struct rte_eth_txconf *tx_conf)
>> {
>> - uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
>> + uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
>> struct virtio_hw *hw = dev->data->dev_private;
>> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
>> + struct virtqueue *vq = hw->vqs[vq_idx];
>> struct virtnet_tx *txvq;
>> uint16_t tx_free_thresh;
>>
>> @@ -823,9 +823,9 @@ int
>> virtio_dev_tx_queue_setup_finish(struct rte_eth_dev *dev,
>> uint16_t queue_idx)
>> {
>> - uint8_t vtpci_queue_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
>> + uint8_t vq_idx = 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX;
>> struct virtio_hw *hw = dev->data->dev_private;
>> - struct virtqueue *vq = hw->vqs[vtpci_queue_idx];
>> + struct virtqueue *vq = hw->vqs[vq_idx];
>>
>> PMD_INIT_FUNC_TRACE();
>>
>> diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> b/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> index c272766a9f..a3dcc01a43 100644
>> --- a/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.c
>> @@ -12,7 +12,7 @@
>>
>> #include "virtio_logs.h"
>> #include "virtio_ethdev.h"
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> #include "virtqueue.h"
>>
>> #define BYTE_SIZE 8
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index 210a3704e7..9d2a8505b3 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -9,7 +9,9 @@
>> #include <linux/types.h>
>> #include <linux/ioctl.h>
>>
>> -#include "../virtio_pci.h"
>> +#include <rte_errno.h>
>> +
>> +#include "../virtio.h"
>> #include "../virtio_logs.h"
>> #include "../virtqueue.h"
>>
>> diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c
>> index 59a2cb6599..1f9af3c31b 100644
>> --- a/drivers/net/virtio/virtqueue.c
>> +++ b/drivers/net/virtio/virtqueue.c
>> @@ -7,7 +7,7 @@
>>
>> #include "virtqueue.h"
>> #include "virtio_logs.h"
>> -#include "virtio_pci.h"
>> +#include "virtio.h"
>> #include "virtio_rxtx_simple.h"
>>
>> /*
>> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
>> index 6c1df6f8e5..9274c48080 100644
>> --- a/drivers/net/virtio/virtqueue.h
>> +++ b/drivers/net/virtio/virtqueue.h
>> @@ -449,11 +449,11 @@ virtqueue_full(const struct virtqueue *vq)
>> }
>>
>> static inline int
>> -virtio_get_queue_type(struct virtio_hw *hw, uint16_t vtpci_queue_idx)
>> +virtio_get_queue_type(struct virtio_hw *hw, uint16_t vq_idx)
>> {
>> - if (vtpci_queue_idx == hw->max_queue_pairs * 2)
>> + if (vq_idx == hw->max_queue_pairs * 2)
>> return VTNET_CQ;
>> - else if (vtpci_queue_idx % 2 == 0)
>> + else if (vq_idx % 2 == 0)
>> return VTNET_RQ;
>> else
>> return VTNET_TQ;
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
2020-12-20 21:13 ` [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code Maxime Coquelin
2020-12-30 3:25 ` Xia, Chenbo
@ 2021-01-06 16:18 ` David Marchand
2021-01-15 11:10 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: David Marchand @ 2021-01-06 16:18 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This patch finalizes the bus isolation part of this
> refactoring.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
[snip]
> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
> index 6c1df6f8e5..9274c48080 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -449,11 +449,11 @@ virtqueue_full(const struct virtqueue *vq)
> }
>
> static inline int
> -virtio_get_queue_type(struct virtio_hw *hw, uint16_t vtpci_queue_idx)
> +virtio_get_queue_type(struct virtio_hw *hw, uint16_t vq_idx)
> {
> - if (vtpci_queue_idx == hw->max_queue_pairs * 2)
> + if (vq_idx == hw->max_queue_pairs * 2)
> return VTNET_CQ;
> - else if (vtpci_queue_idx % 2 == 0)
> + else if (vq_idx % 2 == 0)
> return VTNET_RQ;
> else
> return VTNET_TQ;
> --
> 2.29.2
>
I noticed:
drivers/net/virtio/virtqueue.h: uint16_t vq_queue_index; /**< PCI
queue index */
Worth cleaning while at it?
Reviewed-by: David Marchand <david.marchand@redhat.com>
Same comment as Chenbo, this is a great cleanup so far.
I'll look at the rest of the series probably tomorrow.
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code
2021-01-06 16:18 ` David Marchand
@ 2021-01-15 11:10 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 11:10 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On 1/6/21 5:18 PM, David Marchand wrote:
> On Sun, Dec 20, 2020 at 10:15 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>>
>> This patch finalizes the bus isolation part of this
>> refactoring.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>
> [snip]
>
>> diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h
>> index 6c1df6f8e5..9274c48080 100644
>> --- a/drivers/net/virtio/virtqueue.h
>> +++ b/drivers/net/virtio/virtqueue.h
>> @@ -449,11 +449,11 @@ virtqueue_full(const struct virtqueue *vq)
>> }
>>
>> static inline int
>> -virtio_get_queue_type(struct virtio_hw *hw, uint16_t vtpci_queue_idx)
>> +virtio_get_queue_type(struct virtio_hw *hw, uint16_t vq_idx)
>> {
>> - if (vtpci_queue_idx == hw->max_queue_pairs * 2)
>> + if (vq_idx == hw->max_queue_pairs * 2)
>> return VTNET_CQ;
>> - else if (vtpci_queue_idx % 2 == 0)
>> + else if (vq_idx % 2 == 0)
>> return VTNET_RQ;
>> else
>> return VTNET_TQ;
>> --
>> 2.29.2
>>
>
> I noticed:
> drivers/net/virtio/virtqueue.h: uint16_t vq_queue_index; /**< PCI
> queue index */
>
> Worth cleaning while at it?
I removed the comment, it is self-explanatory.
Thanks,
Maxime
> Reviewed-by: David Marchand <david.marchand@redhat.com>
>
>
>
> Same comment as Chenbo, this is a great cleanup so far.
> I'll look at the rest of the series probably tomorrow.
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (21 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 22/40] net/virtio: remove last PCI refs in non-PCI code Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 11:50 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 24/40] net/virtio: add Virtio-user ops to set owner Maxime Coquelin
` (17 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch makes vhost_user_write() consistent with
vhost_user_read(), by passing a pointer on the Vhost-user
message instead of a buffer pointer and its length, which
is now calculated in the function.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost_user.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index b93e65c60b..e9053eea95 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -51,7 +51,7 @@ struct vhost_user_msg {
(sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
static int
-vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
+vhost_user_write(int fd, struct vhost_user_msg *msg, int *fds, int fd_num)
{
int r;
struct msghdr msgh;
@@ -63,8 +63,8 @@ vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
memset(&msgh, 0, sizeof(msgh));
memset(control, 0, sizeof(control));
- iov.iov_base = (uint8_t *)buf;
- iov.iov_len = len;
+ iov.iov_base = (uint8_t *)msg;
+ iov.iov_len = VHOST_USER_HDR_SIZE + msg->size;
msgh.msg_iov = &iov;
msgh.msg_iovlen = 1;
@@ -259,7 +259,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
int has_reply_ack = 0;
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
- int len;
int vhostfd = dev->vhostfd;
RTE_SET_USED(m);
@@ -359,8 +358,7 @@ vhost_user_sock(struct virtio_user_dev *dev,
return -1;
}
- len = VHOST_USER_HDR_SIZE + msg.size;
- if (vhost_user_write(vhostfd, &msg, len, fds, fd_num) < 0) {
+ if (vhost_user_write(vhostfd, &msg, fds, fd_num) < 0) {
PMD_DRV_LOG(ERR, "%s failed: %s",
vhost_msg_strings[req], strerror(errno));
return -1;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent
2020-12-20 21:13 ` [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent Maxime Coquelin
@ 2021-01-06 11:50 ` Xia, Chenbo
2021-01-15 9:47 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 11:50 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 23/40] net/virtio: make Vhost-user req sender consistent
>
> This patch makes vhost_user_write() consistent with
> vhost_user_read(), by passing a pointer on the Vhost-user
You mean a pointer 'to' the vhost-user msg? :P
Thanks,
Chenbo
> message instead of a buffer pointer and its length, which
> is now calculated in the function.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent
2021-01-06 11:50 ` Xia, Chenbo
@ 2021-01-15 9:47 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 9:47 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 12:50 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 23/40] net/virtio: make Vhost-user req sender consistent
>>
>> This patch makes vhost_user_write() consistent with
>> vhost_user_read(), by passing a pointer on the Vhost-user
>
> You mean a pointer 'to' the vhost-user msg? :P
Changed to:
"by passing a Vhost-user message pointer instead of a buffer pointer and
its length"
Thanks,
Maxime
> Thanks,
> Chenbo
>
>> message instead of a buffer pointer and its length, which
>> is now calculated in the function.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 24/40] net/virtio: add Virtio-user ops to set owner
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (22 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 23/40] net/virtio: make Vhost-user req sender consistent Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 11:50 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops Maxime Coquelin
` (16 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch implements a dedicated callabck for
sending owner request. All the requests will be
converted that way so that backends other than
Vhost-user don't have to work around being it.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 1 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 28 +++++++++++++++--
drivers/net/virtio/virtio_user/vhost_user.c | 21 +++++++++++--
drivers/net/virtio/virtio_user/vhost_vdpa.c | 30 ++++++++++++++++---
.../net/virtio/virtio_user/virtio_user_dev.c | 3 +-
5 files changed, 72 insertions(+), 11 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 9d2a8505b3..8e819ecfb8 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -101,6 +101,7 @@ struct virtio_user_dev;
struct virtio_user_backend_ops {
int (*setup)(struct virtio_user_dev *dev);
+ int (*set_owner)(struct virtio_user_dev *dev);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 2c805077af..b79dcad179 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -6,6 +6,7 @@
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
+#include <errno.h>
#include <rte_memory.h>
@@ -55,8 +56,28 @@ get_vhost_kernel_max_regions(void)
close(fd);
}
+static int
+vhost_kernel_ioctl(int fd, uint64_t request, void *arg)
+{
+ int ret;
+
+ ret = ioctl(fd, request, arg);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Vhost-kernel ioctl %"PRIu64" failed (%s)",
+ request, strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_owner(struct virtio_user_dev *dev)
+{
+ return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
- [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
[VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
[VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
@@ -175,7 +196,7 @@ tap_support_features(void)
}
static int
-vhost_kernel_ioctl(struct virtio_user_dev *dev,
+vhost_kernel_send_request(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
{
@@ -385,6 +406,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_kernel = {
.setup = vhost_kernel_setup,
- .send_request = vhost_kernel_ioctl,
+ .set_owner = vhost_kernel_set_owner,
+ .send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index e9053eea95..5ab15318ac 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -125,6 +125,24 @@ vhost_user_read(int fd, struct vhost_user_msg *msg)
return -1;
}
+static int
+vhost_user_set_owner(struct virtio_user_dev *dev)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_OWNER,
+ .flags = VHOST_USER_VERSION,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set owner");
+ return -1;
+ }
+
+ return 0;
+}
+
struct walk_arg {
struct vhost_memory *vm;
int *fds;
@@ -230,7 +248,6 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_SET_OWNER",
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
[VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
[VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
@@ -303,7 +320,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
msg.size = sizeof(m.payload.u64);
break;
- case VHOST_USER_SET_OWNER:
case VHOST_USER_RESET_OWNER:
break;
@@ -514,6 +530,7 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_user = {
.setup = vhost_user_setup,
+ .set_owner = vhost_user_set_owner,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index c7b9349fc8..f405b6b86a 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -37,7 +37,6 @@
struct vhost_vring_state)
static uint64_t vhost_req_user_to_vdpa[] = {
- [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
[VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
[VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
@@ -80,6 +79,28 @@ struct vhost_msg {
};
};
+
+static int
+vhost_vdpa_ioctl(int fd, uint64_t request, void *arg)
+{
+ int ret;
+
+ ret = ioctl(fd, request, arg);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Vhost-vDPA ioctl %"PRIu64" failed (%s)",
+ request, strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_vdpa_set_owner(struct virtio_user_dev *dev)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_OWNER, NULL);
+}
+
static int
vhost_vdpa_dma_map(struct virtio_user_dev *dev, void *addr,
uint64_t iova, size_t len)
@@ -192,7 +213,7 @@ vhost_vdpa_dma_map_all(struct virtio_user_dev *dev)
(1ULL << VIRTIO_NET_F_CSUM))
static int
-vhost_vdpa_ioctl(struct virtio_user_dev *dev,
+vhost_vdpa_send_request(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
{
@@ -280,7 +301,7 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
.num = enable,
};
- if (vhost_vdpa_ioctl(dev, VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_vdpa_send_request(dev, VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
@@ -291,7 +312,8 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_vdpa = {
.setup = vhost_vdpa_setup,
- .send_request = vhost_vdpa_ioctl,
+ .set_owner = vhost_vdpa_set_owner,
+ .send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
.dma_unmap = vhost_vdpa_dma_unmap,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 053f0267ca..f8e4581951 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -482,8 +482,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
(1ULL << VHOST_USER_F_PROTOCOL_FEATURES);
if (!dev->is_server) {
- if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER,
- NULL) < 0) {
+ if (dev->ops->set_owner(dev) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s",
strerror(errno));
return -1;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 24/40] net/virtio: add Virtio-user ops to set owner
2020-12-20 21:13 ` [dpdk-dev] [PATCH 24/40] net/virtio: add Virtio-user ops to set owner Maxime Coquelin
@ 2021-01-06 11:50 ` Xia, Chenbo
0 siblings, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 11:50 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 24/40] net/virtio: add Virtio-user ops to set owner
>
> This patch implements a dedicated callabck for
s/callabck/callback
Thanks,
Chenbo
> sending owner request. All the requests will be
> converted that way so that backends other than
> Vhost-user don't have to work around being it.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (23 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 24/40] net/virtio: add Virtio-user ops to set owner Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 11:54 ` Xia, Chenbo
2021-01-13 13:57 ` Adrian Moreno
2020-12-20 21:13 ` [dpdk-dev] [PATCH 26/40] net/virtio: add Virtio-user protocol " Maxime Coquelin
` (15 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduce new callbacks for getting
and setting Virtio features, and implements them
for the different backend types.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
.../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
.../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
.../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
drivers/net/virtio/virtio_user_ethdev.c | 3 +-
8 files changed, 188 insertions(+), 97 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 8e819ecfb8..16978e27ed 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -102,6 +102,8 @@ struct virtio_user_dev;
struct virtio_user_backend_ops {
int (*setup)(struct virtio_user_dev *dev);
int (*set_owner)(struct virtio_user_dev *dev);
+ int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
+ int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index b79dcad179..f44df8ef1f 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -38,6 +38,28 @@ struct vhost_memory_kernel {
#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+/* with below features, vhost kernel does not need to do the checksum and TSO,
+ * these info will be passed to virtio_user through virtio net header.
+ */
+#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
+ (1ULL << VIRTIO_NET_F_GUEST_UFO))
+
+/* with below features, when flows from virtio_user to vhost kernel
+ * (1) if flows goes up through the kernel networking stack, it does not need
+ * to verify checksum, which can save CPU cycles;
+ * (2) if flows goes through a Linux bridge and outside from an interface
+ * (kernel driver), checksum and TSO will be done by GSO in kernel or even
+ * offloaded into real physical device.
+ */
+#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_CSUM))
+
static uint64_t max_regions = 64;
static void
@@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
}
+static int
+vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
+{
+ int ret;
+ unsigned int tap_features;
+
+ ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to get features");
+ return -1;
+ }
+
+ ret = tap_support_features(&tap_features);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to get TAP features)");
+ return -1;
+ }
+
+ /* with tap as the backend, all these features are supported
+ * but not claimed by vhost-net, so we add them back when
+ * reporting to upper layer.
+ */
+ if (tap_features & IFF_VNET_HDR) {
+ *features |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *features |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
+ /* vhost_kernel will not declare this feature, but it does
+ * support multi-queue.
+ */
+ if (tap_features & IFF_MULTI_QUEUE)
+ *features |= (1ull << VIRTIO_NET_F_MQ);
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_features(struct virtio_user_dev *dev, uint64_t features)
+{
+ /* We don't need memory protection here */
+ features &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ /* VHOST kernel does not know about below flags */
+ features &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ features &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ features &= ~(1ULL << VIRTIO_NET_F_MQ);
+
+ return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_FEATURES, &features);
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
- [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
[VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
[VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
[VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
@@ -150,51 +219,6 @@ prepare_vhost_memory_kernel(void)
return vm;
}
-/* with below features, vhost kernel does not need to do the checksum and TSO,
- * these info will be passed to virtio_user through virtio net header.
- */
-#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
- ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
- (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
- (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
- (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
- (1ULL << VIRTIO_NET_F_GUEST_UFO))
-
-/* with below features, when flows from virtio_user to vhost kernel
- * (1) if flows goes up through the kernel networking stack, it does not need
- * to verify checksum, which can save CPU cycles;
- * (2) if flows goes through a Linux bridge and outside from an interface
- * (kernel driver), checksum and TSO will be done by GSO in kernel or even
- * offloaded into real physical device.
- */
-#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
- ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
- (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
- (1ULL << VIRTIO_NET_F_CSUM))
-
-static unsigned int
-tap_support_features(void)
-{
- int tapfd;
- unsigned int tap_features;
-
- tapfd = open(PATH_NET_TUN, O_RDWR);
- if (tapfd < 0) {
- PMD_DRV_LOG(ERR, "fail to open %s: %s",
- PATH_NET_TUN, strerror(errno));
- return -1;
- }
-
- if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
- PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
- close(tapfd);
- return -1;
- }
-
- close(tapfd);
- return tap_features;
-}
-
static int
vhost_kernel_send_request(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -206,7 +230,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
struct vhost_memory_kernel *vm = NULL;
int vhostfd;
unsigned int queue_sel;
- unsigned int features;
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -219,17 +242,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
arg = (void *)vm;
}
- if (req_kernel == VHOST_SET_FEATURES) {
- /* We don't need memory protection here */
- *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
-
- /* VHOST kernel does not know about below flags */
- *(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
- *(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
-
- *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
- }
-
switch (req_kernel) {
case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
@@ -259,24 +271,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
ret = ioctl(vhostfd, req_kernel, arg);
}
- if (!ret && req_kernel == VHOST_GET_FEATURES) {
- features = tap_support_features();
- /* with tap as the backend, all these features are supported
- * but not claimed by vhost-net, so we add them back when
- * reporting to upper layer.
- */
- if (features & IFF_VNET_HDR) {
- *((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
- *((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
- }
-
- /* vhost_kernel will not declare this feature, but it does
- * support multi-queue.
- */
- if (features & IFF_MULTI_QUEUE)
- *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
- }
-
if (vm)
free(vm);
@@ -407,6 +401,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_kernel = {
.setup = vhost_kernel_setup,
.set_owner = vhost_kernel_set_owner,
+ .get_features = vhost_kernel_get_features,
+ .set_features = vhost_kernel_set_features,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index eade702c5c..99096bdf39 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -18,6 +18,29 @@
#include "../virtio_logs.h"
#include "../virtio.h"
+
+int
+tap_support_features(unsigned int *tap_features)
+{
+ int tapfd;
+
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(tapfd, TUNGETFEATURES, tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ close(tapfd);
+ return -1;
+ }
+
+ close(tapfd);
+ return 0;
+}
+
int
vhost_kernel_tap_set_offload(int fd, uint64_t features)
{
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
index 5c4447b296..ed03fce21e 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -43,5 +43,6 @@ int vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq,
const char *mac, uint64_t features);
int vhost_kernel_tap_set_offload(int fd, uint64_t features);
int vhost_kernel_tap_set_queue(int fd, bool attach);
+int tap_support_features(unsigned int *tap_features);
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 5ab15318ac..d204fa1eb0 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -143,6 +143,62 @@ vhost_user_set_owner(struct virtio_user_dev *dev)
return 0;
}
+static int
+vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_GET_FEATURES,
+ .flags = VHOST_USER_VERSION,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0)
+ goto err;
+
+ ret = vhost_user_read(dev->vhostfd, &msg);
+ if (ret < 0)
+ goto err;
+
+ if (msg.request != VHOST_USER_GET_FEATURES) {
+ PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
+ goto err;
+ }
+
+ if (msg.size != sizeof(*features)) {
+ PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
+ goto err;
+ }
+
+ *features = msg.payload.u64;
+
+ return 0;
+err:
+ PMD_DRV_LOG(ERR, "Failed to get backend features");
+
+ return -1;
+}
+
+static int
+vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_FEATURES,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(features),
+ .payload.u64 = features,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set features");
+ return -1;
+ }
+
+ return 0;
+}
+
struct walk_arg {
struct vhost_memory *vm;
int *fds;
@@ -249,8 +305,6 @@ static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
[VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
[VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
[VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
@@ -299,7 +353,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
(1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
return -ENOTSUP;
/* Fallthrough */
- case VHOST_USER_GET_FEATURES:
case VHOST_USER_GET_PROTOCOL_FEATURES:
need_reply = 1;
break;
@@ -313,7 +366,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
if (has_reply_ack)
msg.flags |= VHOST_USER_NEED_REPLY_MASK;
/* Fallthrough */
- case VHOST_USER_SET_FEATURES:
case VHOST_USER_SET_PROTOCOL_FEATURES:
case VHOST_USER_SET_LOG_BASE:
msg.payload.u64 = *((__u64 *)arg);
@@ -393,7 +445,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
}
switch (req) {
- case VHOST_USER_GET_FEATURES:
case VHOST_USER_GET_STATUS:
case VHOST_USER_GET_PROTOCOL_FEATURES:
if (msg.size != sizeof(m.payload.u64)) {
@@ -531,6 +582,8 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_user = {
.setup = vhost_user_setup,
.set_owner = vhost_user_set_owner,
+ .get_features = vhost_user_get_features,
+ .set_features = vhost_user_set_features,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index f405b6b86a..c0a9b5b767 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -38,8 +38,6 @@
static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
- [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
[VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
[VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
[VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
@@ -101,6 +99,32 @@ vhost_vdpa_set_owner(struct virtio_user_dev *dev)
return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_OWNER, NULL);
}
+static int
+vhost_vdpa_get_features(struct virtio_user_dev *dev, uint64_t *features)
+{
+ int ret;
+
+ ret = vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_FEATURES, features);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Failed to get features");
+ return -1;
+ }
+
+ /* Multiqueue not supported for now */
+ *features &= ~(1ULL << VIRTIO_NET_F_MQ);
+
+ return 0;
+}
+
+static int
+vhost_vdpa_set_features(struct virtio_user_dev *dev, uint64_t features)
+{
+ /* WORKAROUND */
+ features |= 1ULL << VIRTIO_F_IOMMU_PLATFORM;
+
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_FEATURES, &features);
+}
+
static int
vhost_vdpa_dma_map(struct virtio_user_dev *dev, void *addr,
uint64_t iova, size_t len)
@@ -227,14 +251,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
if (req_vdpa == VHOST_SET_MEM_TABLE)
return vhost_vdpa_dma_map_all(dev);
- if (req_vdpa == VHOST_SET_FEATURES) {
- /* WORKAROUND */
- *(uint64_t *)arg |= 1ULL << VIRTIO_F_IOMMU_PLATFORM;
-
- /* Multiqueue not supported for now */
- *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
- }
-
switch (req_vdpa) {
case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
@@ -313,6 +329,8 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
struct virtio_user_backend_ops virtio_ops_vdpa = {
.setup = vhost_vdpa_setup,
.set_owner = vhost_vdpa_set_owner,
+ .get_features = vhost_vdpa_get_features,
+ .set_features = vhost_vdpa_set_features,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index f8e4581951..0a85d058a8 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
/* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
features &= ~(1ull << VIRTIO_NET_F_STATUS);
- ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
+ ret = dev->ops->set_features(dev, features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
@@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
return -1;
}
- if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
- &dev->device_features) < 0) {
+ if (dev->ops->get_features(dev, &dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s",
strerror(errno));
return -1;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 283f5c7a36..4d2635c8aa 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
- if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
- &dev->device_features) < 0) {
+ if (dev->ops->get_features(dev, &dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s",
strerror(errno));
return -1;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops Maxime Coquelin
@ 2021-01-06 11:54 ` Xia, Chenbo
2021-01-13 13:43 ` Adrian Moreno
2021-01-13 13:57 ` Adrian Moreno
1 sibling, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 11:54 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 25/40] net/virtio: add Virtio-user features ops
>
> This patch introduce new callbacks for getting
s/introduce/introduces
> and setting Virtio features, and implements them
> for the different backend types.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 2 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
> 8 files changed, 188 insertions(+), 97 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index 8e819ecfb8..16978e27ed 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -102,6 +102,8 @@ struct virtio_user_dev;
> struct virtio_user_backend_ops {
> int (*setup)(struct virtio_user_dev *dev);
> int (*set_owner)(struct virtio_user_dev *dev);
> + int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
> + int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index b79dcad179..f44df8ef1f 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -38,6 +38,28 @@ struct vhost_memory_kernel {
> #define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
> #define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct
> vhost_vring_file)
>
> +/* with below features, vhost kernel does not need to do the checksum and TSO,
> + * these info will be passed to virtio_user through virtio net header.
> + */
> +#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
> + ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
> + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
> + (1ULL << VIRTIO_NET_F_GUEST_UFO))
> +
> +/* with below features, when flows from virtio_user to vhost kernel
> + * (1) if flows goes up through the kernel networking stack, it does not need
> + * to verify checksum, which can save CPU cycles;
> + * (2) if flows goes through a Linux bridge and outside from an interface
> + * (kernel driver), checksum and TSO will be done by GSO in kernel or even
> + * offloaded into real physical device.
> + */
> +#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
> + ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
> + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
> + (1ULL << VIRTIO_NET_F_CSUM))
> +
> static uint64_t max_regions = 64;
>
> static void
> @@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
> return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
> }
>
> +static int
> +vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
> +{
> + int ret;
> + unsigned int tap_features;
> +
> + ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to get features");
> + return -1;
> + }
> +
> + ret = tap_support_features(&tap_features);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to get TAP features)");
should delete ')' after 'features'?
> + return -1;
> + }
> +
> + /* with tap as the backend, all these features are supported
> + * but not claimed by vhost-net, so we add them back when
> + * reporting to upper layer.
> + */
<snip>
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index f8e4581951..0a85d058a8 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
> /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
> features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
> features &= ~(1ull << VIRTIO_NET_F_STATUS);
> - ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
> + ret = dev->ops->set_features(dev, features);
I noticed that virtio_user_dev_set_features is called by virtio_user_set_status.
The former may fail but the latter will ignore the failure. So this will happen:
setting features already failed but virtio-user still continue to do things. IMHO,
this is not very good (similar things may happen for virtio_user_start_device).
What do you think?
Thanks,
Chenbo
> if (ret < 0)
> goto error;
> PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
> @@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
> *path, int queues,
> return -1;
> }
>
> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> - &dev->device_features) < 0) {
> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> PMD_INIT_LOG(ERR, "get_features failed: %s",
> strerror(errno));
> return -1;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
> b/drivers/net/virtio/virtio_user_ethdev.c
> index 283f5c7a36..4d2635c8aa 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>
> virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>
> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> - &dev->device_features) < 0) {
> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> PMD_INIT_LOG(ERR, "get_features failed: %s",
> strerror(errno));
> return -1;
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2021-01-06 11:54 ` Xia, Chenbo
@ 2021-01-13 13:43 ` Adrian Moreno
2021-01-13 13:54 ` Maxime Coquelin
2021-01-15 14:19 ` Maxime Coquelin
0 siblings, 2 replies; 149+ messages in thread
From: Adrian Moreno @ 2021-01-13 13:43 UTC (permalink / raw)
To: Xia, Chenbo, Maxime Coquelin, dev, olivier.matz, david.marchand
Hi Chenbo
On 1/6/21 12:54 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 25/40] net/virtio: add Virtio-user features ops
>>
>> This patch introduce new callbacks for getting
>
> s/introduce/introduces
>
>> and setting Virtio features, and implements them
>> for the different backend types.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
>> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
>> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
>> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
>> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
>> 8 files changed, 188 insertions(+), 97 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index 8e819ecfb8..16978e27ed 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -102,6 +102,8 @@ struct virtio_user_dev;
>> struct virtio_user_backend_ops {
>> int (*setup)(struct virtio_user_dev *dev);
>> int (*set_owner)(struct virtio_user_dev *dev);
>> + int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
>> + int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
>> int (*send_request)(struct virtio_user_dev *dev,
>> enum vhost_user_request req,
>> void *arg);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index b79dcad179..f44df8ef1f 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -38,6 +38,28 @@ struct vhost_memory_kernel {
>> #define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
>> #define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct
>> vhost_vring_file)
>>
>> +/* with below features, vhost kernel does not need to do the checksum and TSO,
>> + * these info will be passed to virtio_user through virtio net header.
>> + */
>> +#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
>> + ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
>> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
>> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
>> + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
>> + (1ULL << VIRTIO_NET_F_GUEST_UFO))
>> +
>> +/* with below features, when flows from virtio_user to vhost kernel
>> + * (1) if flows goes up through the kernel networking stack, it does not need
>> + * to verify checksum, which can save CPU cycles;
>> + * (2) if flows goes through a Linux bridge and outside from an interface
>> + * (kernel driver), checksum and TSO will be done by GSO in kernel or even
>> + * offloaded into real physical device.
>> + */
>> +#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
>> + ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
>> + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
>> + (1ULL << VIRTIO_NET_F_CSUM))
>> +
>> static uint64_t max_regions = 64;
>>
>> static void
>> @@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
>> return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
>> }
>>
>> +static int
>> +vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
>> +{
>> + int ret;
>> + unsigned int tap_features;
>> +
>> + ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to get features");
>> + return -1;
>> + }
>> +
>> + ret = tap_support_features(&tap_features);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to get TAP features)");
>
> should delete ')' after 'features'?
>
>> + return -1;
>> + }
>> +
>> + /* with tap as the backend, all these features are supported
>> + * but not claimed by vhost-net, so we add them back when
>> + * reporting to upper layer.
>> + */
>
> <snip>
>
>> .dma_map = vhost_vdpa_dma_map,
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index f8e4581951..0a85d058a8 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>> /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
>> features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
>> features &= ~(1ull << VIRTIO_NET_F_STATUS);
>> - ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
>> + ret = dev->ops->set_features(dev, features);
>
> I noticed that virtio_user_dev_set_features is called by virtio_user_set_status.
> The former may fail but the latter will ignore the failure. So this will happen:
> setting features already failed but virtio-user still continue to do things. IMHO,
> this is not very good (similar things may happen for virtio_user_start_device).
> What do you think?
>
You're right. Although set_status(features_ok) should be followed by
get_status() to check if device has set feature_ok bit, the problem is that
set_status is called on device initialization, which would happen even when the
vhost backed is not yet connected (server-mode). Failing there would tear down
the ethdev which would make the future "hotplug" fail. Hopefully this will all
be fixed by Maxime's refactoring :)
So +1 to reconsider improving error propagation here.
> Thanks,
> Chenbo
>
>> if (ret < 0)
>> goto error;
>> PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
>> @@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>> *path, int queues,
>> return -1;
>> }
>>
>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>> - &dev->device_features) < 0) {
>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>> strerror(errno));
>> return -1;
>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
>> b/drivers/net/virtio/virtio_user_ethdev.c
>> index 283f5c7a36..4d2635c8aa 100644
>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>> @@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>>
>> virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>>
>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>> - &dev->device_features) < 0) {
>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>> strerror(errno));
>> return -1;
>> --
>> 2.29.2
>
--
Adrián Moreno
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2021-01-13 13:43 ` Adrian Moreno
@ 2021-01-13 13:54 ` Maxime Coquelin
2021-01-15 14:19 ` Maxime Coquelin
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-13 13:54 UTC (permalink / raw)
To: Adrian Moreno, Xia, Chenbo, dev, olivier.matz, david.marchand
On 1/13/21 2:43 PM, Adrian Moreno wrote:
> Hi Chenbo
>
> On 1/6/21 12:54 PM, Xia, Chenbo wrote:
>> Hi Maxime,
>>
>>> -----Original Message-----
>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> Sent: Monday, December 21, 2020 5:14 AM
>>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>>> amorenoz@redhat.com; david.marchand@redhat.com
>>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> Subject: [PATCH 25/40] net/virtio: add Virtio-user features ops
>>>
>>> This patch introduce new callbacks for getting
>>
>> s/introduce/introduces
>>
>>> and setting Virtio features, and implements them
>>> for the different backend types.
>>>
>>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> ---
>>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>>> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
>>> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
>>> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
>>> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
>>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
>>> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
>>> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
>>> 8 files changed, 188 insertions(+), 97 deletions(-)
>>>
>>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>>> b/drivers/net/virtio/virtio_user/vhost.h
>>> index 8e819ecfb8..16978e27ed 100644
>>> --- a/drivers/net/virtio/virtio_user/vhost.h
>>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>>> @@ -102,6 +102,8 @@ struct virtio_user_dev;
>>> struct virtio_user_backend_ops {
>>> int (*setup)(struct virtio_user_dev *dev);
>>> int (*set_owner)(struct virtio_user_dev *dev);
>>> + int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
>>> + int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
>>> int (*send_request)(struct virtio_user_dev *dev,
>>> enum vhost_user_request req,
>>> void *arg);
>>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> index b79dcad179..f44df8ef1f 100644
>>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> @@ -38,6 +38,28 @@ struct vhost_memory_kernel {
>>> #define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
>>> #define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct
>>> vhost_vring_file)
>>>
>>> +/* with below features, vhost kernel does not need to do the checksum and TSO,
>>> + * these info will be passed to virtio_user through virtio net header.
>>> + */
>>> +#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
>>> + ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_UFO))
>>> +
>>> +/* with below features, when flows from virtio_user to vhost kernel
>>> + * (1) if flows goes up through the kernel networking stack, it does not need
>>> + * to verify checksum, which can save CPU cycles;
>>> + * (2) if flows goes through a Linux bridge and outside from an interface
>>> + * (kernel driver), checksum and TSO will be done by GSO in kernel or even
>>> + * offloaded into real physical device.
>>> + */
>>> +#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
>>> + ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
>>> + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
>>> + (1ULL << VIRTIO_NET_F_CSUM))
>>> +
>>> static uint64_t max_regions = 64;
>>>
>>> static void
>>> @@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
>>> return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
>>> }
>>>
>>> +static int
>>> +vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
>>> +{
>>> + int ret;
>>> + unsigned int tap_features;
>>> +
>>> + ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
>>> + if (ret < 0) {
>>> + PMD_DRV_LOG(ERR, "Failed to get features");
>>> + return -1;
>>> + }
>>> +
>>> + ret = tap_support_features(&tap_features);
>>> + if (ret < 0) {
>>> + PMD_DRV_LOG(ERR, "Failed to get TAP features)");
>>
>> should delete ')' after 'features'?
>>
>>> + return -1;
>>> + }
>>> +
>>> + /* with tap as the backend, all these features are supported
>>> + * but not claimed by vhost-net, so we add them back when
>>> + * reporting to upper layer.
>>> + */
>>
>> <snip>
>>
>>> .dma_map = vhost_vdpa_dma_map,
>>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> index f8e4581951..0a85d058a8 100644
>>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> @@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>>> /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
>>> features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
>>> features &= ~(1ull << VIRTIO_NET_F_STATUS);
>>> - ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
>>> + ret = dev->ops->set_features(dev, features);
>>
>> I noticed that virtio_user_dev_set_features is called by virtio_user_set_status.
>> The former may fail but the latter will ignore the failure. So this will happen:
>> setting features already failed but virtio-user still continue to do things. IMHO,
>> this is not very good (similar things may happen for virtio_user_start_device).
>> What do you think?
>>
> You're right. Although set_status(features_ok) should be followed by
> get_status() to check if device has set feature_ok bit, the problem is that
> set_status is called on device initialization, which would happen even when the
> vhost backed is not yet connected (server-mode). Failing there would tear down
> the ethdev which would make the future "hotplug" fail. Hopefully this will all
> be fixed by Maxime's refactoring :)
> So +1 to reconsider improving error propagation here.
Yes, makes sense.
Sorry for not replying earlier, I'm rebasing the series and than will
look at all your comments one by one.
Thanks,
Maxime
>> Thanks,
>> Chenbo
>>
>>> if (ret < 0)
>>> goto error;
>>> PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
>>> @@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>>> *path, int queues,
>>> return -1;
>>> }
>>>
>>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>>> - &dev->device_features) < 0) {
>>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>>> strerror(errno));
>>> return -1;
>>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
>>> b/drivers/net/virtio/virtio_user_ethdev.c
>>> index 283f5c7a36..4d2635c8aa 100644
>>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>>> @@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>>>
>>> virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>>>
>>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>>> - &dev->device_features) < 0) {
>>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>>> strerror(errno));
>>> return -1;
>>> --
>>> 2.29.2
>>
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2021-01-13 13:43 ` Adrian Moreno
2021-01-13 13:54 ` Maxime Coquelin
@ 2021-01-15 14:19 ` Maxime Coquelin
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 14:19 UTC (permalink / raw)
To: Adrian Moreno, Xia, Chenbo, dev, olivier.matz, david.marchand
On 1/13/21 2:43 PM, Adrian Moreno wrote:
> Hi Chenbo
>
> On 1/6/21 12:54 PM, Xia, Chenbo wrote:
>> Hi Maxime,
>>
>>> -----Original Message-----
>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> Sent: Monday, December 21, 2020 5:14 AM
>>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>>> amorenoz@redhat.com; david.marchand@redhat.com
>>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> Subject: [PATCH 25/40] net/virtio: add Virtio-user features ops
>>>
>>> This patch introduce new callbacks for getting
>>
>> s/introduce/introduces
>>
>>> and setting Virtio features, and implements them
>>> for the different backend types.
>>>
>>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>>> ---
>>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>>> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
>>> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
>>> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
>>> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
>>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
>>> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
>>> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
>>> 8 files changed, 188 insertions(+), 97 deletions(-)
>>>
>>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>>> b/drivers/net/virtio/virtio_user/vhost.h
>>> index 8e819ecfb8..16978e27ed 100644
>>> --- a/drivers/net/virtio/virtio_user/vhost.h
>>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>>> @@ -102,6 +102,8 @@ struct virtio_user_dev;
>>> struct virtio_user_backend_ops {
>>> int (*setup)(struct virtio_user_dev *dev);
>>> int (*set_owner)(struct virtio_user_dev *dev);
>>> + int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
>>> + int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
>>> int (*send_request)(struct virtio_user_dev *dev,
>>> enum vhost_user_request req,
>>> void *arg);
>>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> index b79dcad179..f44df8ef1f 100644
>>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>>> @@ -38,6 +38,28 @@ struct vhost_memory_kernel {
>>> #define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
>>> #define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct
>>> vhost_vring_file)
>>>
>>> +/* with below features, vhost kernel does not need to do the checksum and TSO,
>>> + * these info will be passed to virtio_user through virtio net header.
>>> + */
>>> +#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
>>> + ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
>>> + (1ULL << VIRTIO_NET_F_GUEST_UFO))
>>> +
>>> +/* with below features, when flows from virtio_user to vhost kernel
>>> + * (1) if flows goes up through the kernel networking stack, it does not need
>>> + * to verify checksum, which can save CPU cycles;
>>> + * (2) if flows goes through a Linux bridge and outside from an interface
>>> + * (kernel driver), checksum and TSO will be done by GSO in kernel or even
>>> + * offloaded into real physical device.
>>> + */
>>> +#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
>>> + ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
>>> + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
>>> + (1ULL << VIRTIO_NET_F_CSUM))
>>> +
>>> static uint64_t max_regions = 64;
>>>
>>> static void
>>> @@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
>>> return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
>>> }
>>>
>>> +static int
>>> +vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
>>> +{
>>> + int ret;
>>> + unsigned int tap_features;
>>> +
>>> + ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
>>> + if (ret < 0) {
>>> + PMD_DRV_LOG(ERR, "Failed to get features");
>>> + return -1;
>>> + }
>>> +
>>> + ret = tap_support_features(&tap_features);
>>> + if (ret < 0) {
>>> + PMD_DRV_LOG(ERR, "Failed to get TAP features)");
>>
>> should delete ')' after 'features'?
>>
>>> + return -1;
>>> + }
>>> +
>>> + /* with tap as the backend, all these features are supported
>>> + * but not claimed by vhost-net, so we add them back when
>>> + * reporting to upper layer.
>>> + */
>>
>> <snip>
>>
>>> .dma_map = vhost_vdpa_dma_map,
>>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> index f8e4581951..0a85d058a8 100644
>>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>>> @@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>>> /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
>>> features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
>>> features &= ~(1ull << VIRTIO_NET_F_STATUS);
>>> - ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
>>> + ret = dev->ops->set_features(dev, features);
>>
>> I noticed that virtio_user_dev_set_features is called by virtio_user_set_status.
>> The former may fail but the latter will ignore the failure. So this will happen:
>> setting features already failed but virtio-user still continue to do things. IMHO,
>> this is not very good (similar things may happen for virtio_user_start_device).
>> What do you think?
>>
> You're right. Although set_status(features_ok) should be followed by
> get_status() to check if device has set feature_ok bit, the problem is that
> set_status is called on device initialization, which would happen even when the
> vhost backed is not yet connected (server-mode). Failing there would tear down
> the ethdev which would make the future "hotplug" fail. Hopefully this will all
> be fixed by Maxime's refactoring :)
> So +1 to reconsider improving error propagation here.
I agree with the idea of improving the error propagation, maybe it can
be done after the series is merged as further improvements.
With making server mode made blocking, it will be easier to handle.
But there is still the reconnection case that could be problematic, I
prefer not risking introducing more regressions for this series :).
Is that OK for you?
Cheers,
Maxime
>
>> Thanks,
>> Chenbo
>>
>>> if (ret < 0)
>>> goto error;
>>> PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
>>> @@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>>> *path, int queues,
>>> return -1;
>>> }
>>>
>>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>>> - &dev->device_features) < 0) {
>>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>>> strerror(errno));
>>> return -1;
>>> diff --git a/drivers/net/virtio/virtio_user_ethdev.c
>>> b/drivers/net/virtio/virtio_user_ethdev.c
>>> index 283f5c7a36..4d2635c8aa 100644
>>> --- a/drivers/net/virtio/virtio_user_ethdev.c
>>> +++ b/drivers/net/virtio/virtio_user_ethdev.c
>>> @@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>>>
>>> virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>>>
>>> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
>>> - &dev->device_features) < 0) {
>>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>>> PMD_INIT_LOG(ERR, "get_features failed: %s",
>>> strerror(errno));
>>> return -1;
>>> --
>>> 2.29.2
>>
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops Maxime Coquelin
2021-01-06 11:54 ` Xia, Chenbo
@ 2021-01-13 13:57 ` Adrian Moreno
2021-01-15 14:29 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: Adrian Moreno @ 2021-01-13 13:57 UTC (permalink / raw)
To: Maxime Coquelin, dev, chenbo.xia, olivier.matz, david.marchand
On 12/20/20 10:13 PM, Maxime Coquelin wrote:
> This patch introduce new callbacks for getting
> and setting Virtio features, and implements them
> for the different backend types.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 2 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
> 8 files changed, 188 insertions(+), 97 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
> index 8e819ecfb8..16978e27ed 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -102,6 +102,8 @@ struct virtio_user_dev;
> struct virtio_user_backend_ops {
> int (*setup)(struct virtio_user_dev *dev);
> int (*set_owner)(struct virtio_user_dev *dev);
> + int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
> + int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index b79dcad179..f44df8ef1f 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -38,6 +38,28 @@ struct vhost_memory_kernel {
> #define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
> #define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
>
> +/* with below features, vhost kernel does not need to do the checksum and TSO,
> + * these info will be passed to virtio_user through virtio net header.
> + */
> +#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
> + ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
> + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
> + (1ULL << VIRTIO_NET_F_GUEST_UFO))
> +
> +/* with below features, when flows from virtio_user to vhost kernel
> + * (1) if flows goes up through the kernel networking stack, it does not need
> + * to verify checksum, which can save CPU cycles;
> + * (2) if flows goes through a Linux bridge and outside from an interface
> + * (kernel driver), checksum and TSO will be done by GSO in kernel or even
> + * offloaded into real physical device.
> + */
> +#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
nit: backslash alignment
> + ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
> + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
> + (1ULL << VIRTIO_NET_F_CSUM))
> +
> static uint64_t max_regions = 64;
>
> static void
> @@ -77,10 +99,57 @@ vhost_kernel_set_owner(struct virtio_user_dev *dev)
> return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
> }
>
> +static int
> +vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
> +{
> + int ret;
> + unsigned int tap_features;
> +
> + ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to get features");
> + return -1;
> + }
> +
> + ret = tap_support_features(&tap_features);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to get TAP features)");
> + return -1;
> + }
> +
> + /* with tap as the backend, all these features are supported
> + * but not claimed by vhost-net, so we add them back when
> + * reporting to upper layer.
> + */
> + if (tap_features & IFF_VNET_HDR) {
> + *features |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
> + *features |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
> + }
> +
> + /* vhost_kernel will not declare this feature, but it does
> + * support multi-queue.
> + */
> + if (tap_features & IFF_MULTI_QUEUE)
> + *features |= (1ull << VIRTIO_NET_F_MQ);
> +
> + return 0;
> +}
> +
> +static int
> +vhost_kernel_set_features(struct virtio_user_dev *dev, uint64_t features)
> +{
> + /* We don't need memory protection here */
> + features &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
> + /* VHOST kernel does not know about below flags */
> + features &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
> + features &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
> + features &= ~(1ULL << VIRTIO_NET_F_MQ);
> +
> + return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_FEATURES, &features);
> +}
> +
> static uint64_t vhost_req_user_to_kernel[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
> - [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
> [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
> [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
> [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
> @@ -150,51 +219,6 @@ prepare_vhost_memory_kernel(void)
> return vm;
> }
>
> -/* with below features, vhost kernel does not need to do the checksum and TSO,
> - * these info will be passed to virtio_user through virtio net header.
> - */
> -#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
> - ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
> - (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
> - (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
> - (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
> - (1ULL << VIRTIO_NET_F_GUEST_UFO))
> -
> -/* with below features, when flows from virtio_user to vhost kernel
> - * (1) if flows goes up through the kernel networking stack, it does not need
> - * to verify checksum, which can save CPU cycles;
> - * (2) if flows goes through a Linux bridge and outside from an interface
> - * (kernel driver), checksum and TSO will be done by GSO in kernel or even
> - * offloaded into real physical device.
> - */
> -#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
> - ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
> - (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
> - (1ULL << VIRTIO_NET_F_CSUM))
> -
> -static unsigned int
> -tap_support_features(void)
> -{
> - int tapfd;
> - unsigned int tap_features;
> -
> - tapfd = open(PATH_NET_TUN, O_RDWR);
> - if (tapfd < 0) {
> - PMD_DRV_LOG(ERR, "fail to open %s: %s",
> - PATH_NET_TUN, strerror(errno));
> - return -1;
> - }
> -
> - if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
> - PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
> - close(tapfd);
> - return -1;
> - }
> -
> - close(tapfd);
> - return tap_features;
> -}
> -
> static int
> vhost_kernel_send_request(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> @@ -206,7 +230,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
> struct vhost_memory_kernel *vm = NULL;
> int vhostfd;
> unsigned int queue_sel;
> - unsigned int features;
>
> PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
>
> @@ -219,17 +242,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
> arg = (void *)vm;
> }
>
> - if (req_kernel == VHOST_SET_FEATURES) {
> - /* We don't need memory protection here */
> - *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
> -
> - /* VHOST kernel does not know about below flags */
> - *(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
> - *(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
> -
> - *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
> - }
> -
> switch (req_kernel) {
> case VHOST_SET_VRING_NUM:
> case VHOST_SET_VRING_ADDR:
> @@ -259,24 +271,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
> ret = ioctl(vhostfd, req_kernel, arg);
> }
>
> - if (!ret && req_kernel == VHOST_GET_FEATURES) {
> - features = tap_support_features();
> - /* with tap as the backend, all these features are supported
> - * but not claimed by vhost-net, so we add them back when
> - * reporting to upper layer.
> - */
> - if (features & IFF_VNET_HDR) {
> - *((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
> - *((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
> - }
> -
> - /* vhost_kernel will not declare this feature, but it does
> - * support multi-queue.
> - */
> - if (features & IFF_MULTI_QUEUE)
> - *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
> - }
> -
> if (vm)
> free(vm);
>
> @@ -407,6 +401,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> struct virtio_user_backend_ops virtio_ops_kernel = {
> .setup = vhost_kernel_setup,
> .set_owner = vhost_kernel_set_owner,
> + .get_features = vhost_kernel_get_features,
> + .set_features = vhost_kernel_set_features,
> .send_request = vhost_kernel_send_request,
> .enable_qp = vhost_kernel_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> index eade702c5c..99096bdf39 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
> @@ -18,6 +18,29 @@
> #include "../virtio_logs.h"
> #include "../virtio.h"
>
> +
> +int
> +tap_support_features(unsigned int *tap_features)
> +{
> + int tapfd;
> +
> + tapfd = open(PATH_NET_TUN, O_RDWR);
> + if (tapfd < 0) {
> + PMD_DRV_LOG(ERR, "fail to open %s: %s",
> + PATH_NET_TUN, strerror(errno));
> + return -1;
> + }
> +
> + if (ioctl(tapfd, TUNGETFEATURES, tap_features) == -1) {
> + PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
> + close(tapfd);
> + return -1;
> + }
> +
> + close(tapfd);
> + return 0;
> +}
> +
> int
> vhost_kernel_tap_set_offload(int fd, uint64_t features)
> {
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
> index 5c4447b296..ed03fce21e 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
> @@ -43,5 +43,6 @@ int vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq,
> const char *mac, uint64_t features);
> int vhost_kernel_tap_set_offload(int fd, uint64_t features);
> int vhost_kernel_tap_set_queue(int fd, bool attach);
> +int tap_support_features(unsigned int *tap_features);
>
> #endif
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
> index 5ab15318ac..d204fa1eb0 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -143,6 +143,62 @@ vhost_user_set_owner(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
> +{
> + int ret;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_GET_FEATURES,
> + .flags = VHOST_USER_VERSION,
> + };
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> + if (ret < 0)
> + goto err;
> +
> + ret = vhost_user_read(dev->vhostfd, &msg);
> + if (ret < 0)
> + goto err;
> +
> + if (msg.request != VHOST_USER_GET_FEATURES) {
> + PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
> + goto err;
> + }
> +
> + if (msg.size != sizeof(*features)) {
> + PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
> + goto err;
> + }
> +
> + *features = msg.payload.u64;
> +
> + return 0;
> +err:
> + PMD_DRV_LOG(ERR, "Failed to get backend features");
> +
On possibly socket-related errors, it's sometimes useful to print
sterror(errno). WDYT?
> + return -1;
> +}
> +
> +static int
> +vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
> +{
> + int ret;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_SET_FEATURES,
> + .flags = VHOST_USER_VERSION,
> + .size = sizeof(features),
> + .payload.u64 = features,
> + };
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set features");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> struct walk_arg {
> struct vhost_memory *vm;
> int *fds;
> @@ -249,8 +305,6 @@ static struct vhost_user_msg m;
>
> const char * const vhost_msg_strings[] = {
> [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
> - [VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
> - [VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
> [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
> [VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
> [VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
> @@ -299,7 +353,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
> return -ENOTSUP;
> /* Fallthrough */
> - case VHOST_USER_GET_FEATURES:
> case VHOST_USER_GET_PROTOCOL_FEATURES:
> need_reply = 1;
> break;
> @@ -313,7 +366,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> if (has_reply_ack)
> msg.flags |= VHOST_USER_NEED_REPLY_MASK;
> /* Fallthrough */
> - case VHOST_USER_SET_FEATURES:
> case VHOST_USER_SET_PROTOCOL_FEATURES:
> case VHOST_USER_SET_LOG_BASE:
> msg.payload.u64 = *((__u64 *)arg);
> @@ -393,7 +445,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> }
>
> switch (req) {
> - case VHOST_USER_GET_FEATURES:
> case VHOST_USER_GET_STATUS:
> case VHOST_USER_GET_PROTOCOL_FEATURES:
> if (msg.size != sizeof(m.payload.u64)) {
> @@ -531,6 +582,8 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
> struct virtio_user_backend_ops virtio_ops_user = {
> .setup = vhost_user_setup,
> .set_owner = vhost_user_set_owner,
> + .get_features = vhost_user_get_features,
> + .set_features = vhost_user_set_features,
> .send_request = vhost_user_sock,
> .enable_qp = vhost_user_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index f405b6b86a..c0a9b5b767 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -38,8 +38,6 @@
>
> static uint64_t vhost_req_user_to_vdpa[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
> - [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
> [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
> [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
> [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
> @@ -101,6 +99,32 @@ vhost_vdpa_set_owner(struct virtio_user_dev *dev)
> return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_OWNER, NULL);
> }
>
> +static int
> +vhost_vdpa_get_features(struct virtio_user_dev *dev, uint64_t *features)
> +{
> + int ret;
> +
> + ret = vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_FEATURES, features);
> + if (ret) {
> + PMD_DRV_LOG(ERR, "Failed to get features");
> + return -1;
> + }
> +
> + /* Multiqueue not supported for now */
> + *features &= ~(1ULL << VIRTIO_NET_F_MQ);
> +
> + return 0;
> +}
> +
> +static int
> +vhost_vdpa_set_features(struct virtio_user_dev *dev, uint64_t features)
> +{
> + /* WORKAROUND */
> + features |= 1ULL << VIRTIO_F_IOMMU_PLATFORM;
> +
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_FEATURES, &features);
> +}
> +
> static int
> vhost_vdpa_dma_map(struct virtio_user_dev *dev, void *addr,
> uint64_t iova, size_t len)
> @@ -227,14 +251,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
> if (req_vdpa == VHOST_SET_MEM_TABLE)
> return vhost_vdpa_dma_map_all(dev);
>
> - if (req_vdpa == VHOST_SET_FEATURES) {
> - /* WORKAROUND */
> - *(uint64_t *)arg |= 1ULL << VIRTIO_F_IOMMU_PLATFORM;
> -
> - /* Multiqueue not supported for now */
> - *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
> - }
> -
> switch (req_vdpa) {
> case VHOST_SET_VRING_NUM:
> case VHOST_SET_VRING_ADDR:
> @@ -313,6 +329,8 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
> struct virtio_user_backend_ops virtio_ops_vdpa = {
> .setup = vhost_vdpa_setup,
> .set_owner = vhost_vdpa_set_owner,
> + .get_features = vhost_vdpa_get_features,
> + .set_features = vhost_vdpa_set_features,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index f8e4581951..0a85d058a8 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -141,7 +141,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
> /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
> features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
> features &= ~(1ull << VIRTIO_NET_F_STATUS);
> - ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
> + ret = dev->ops->set_features(dev, features);
> if (ret < 0)
> goto error;
> PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
> @@ -488,8 +488,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
> return -1;
> }
>
> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> - &dev->device_features) < 0) {
> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> PMD_INIT_LOG(ERR, "get_features failed: %s",
> strerror(errno));
> return -1;
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> index 283f5c7a36..4d2635c8aa 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -85,8 +85,7 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>
> virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>
> - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> - &dev->device_features) < 0) {
> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> PMD_INIT_LOG(ERR, "get_features failed: %s",
> strerror(errno));
> return -1;
>
--
Adrián Moreno
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops
2021-01-13 13:57 ` Adrian Moreno
@ 2021-01-15 14:29 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 14:29 UTC (permalink / raw)
To: Adrian Moreno, dev, chenbo.xia, olivier.matz, david.marchand
On 1/13/21 2:57 PM, Adrian Moreno wrote:
>
>
> On 12/20/20 10:13 PM, Maxime Coquelin wrote:
>> This patch introduce new callbacks for getting
>> and setting Virtio features, and implements them
>> for the different backend types.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 150 +++++++++---------
>> .../net/virtio/virtio_user/vhost_kernel_tap.c | 23 +++
>> .../net/virtio/virtio_user/vhost_kernel_tap.h | 1 +
>> drivers/net/virtio/virtio_user/vhost_user.c | 63 +++++++-
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 38 +++--
>> .../net/virtio/virtio_user/virtio_user_dev.c | 5 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 3 +-
>> 8 files changed, 188 insertions(+), 97 deletions(-)
>>
...
>> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
>> index 5ab15318ac..d204fa1eb0 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_user.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
>> @@ -143,6 +143,62 @@ vhost_user_set_owner(struct virtio_user_dev *dev)
>> return 0;
>> }
>>
>> +static int
>> +vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
>> +{
>> + int ret;
>> + struct vhost_user_msg msg = {
>> + .request = VHOST_USER_GET_FEATURES,
>> + .flags = VHOST_USER_VERSION,
>> + };
>> +
>> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
>> + if (ret < 0)
>> + goto err;
>> +
>> + ret = vhost_user_read(dev->vhostfd, &msg);
>> + if (ret < 0)
>> + goto err;
>> +
>> + if (msg.request != VHOST_USER_GET_FEATURES) {
>> + PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
>> + goto err;
>> + }
>> +
>> + if (msg.size != sizeof(*features)) {
>> + PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
>> + goto err;
>> + }
>> +
>> + *features = msg.payload.u64;
>> +
>> + return 0;
>> +err:
>> + PMD_DRV_LOG(ERR, "Failed to get backend features");
>> +
> On possibly socket-related errors, it's sometimes useful to print
> sterror(errno). WDYT?
>> + return -1;
>> +}
I agree, I will do it in vhost_user_read/vhost_user_write.
Actually I have a already done it in vhost_user_write after earlier
comment from Chenbo.
Thanks,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 26/40] net/virtio: add Virtio-user protocol features ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (24 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 25/40] net/virtio: add Virtio-user features ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 11:55 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops Maxime Coquelin
` (14 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduce new callbacks for getting
and setting Vhost-user protocol features.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_user.c | 64 +++++++++++++++++--
.../net/virtio/virtio_user/virtio_user_dev.c | 17 ++---
drivers/net/virtio/virtio_user_ethdev.c | 14 ++--
4 files changed, 69 insertions(+), 28 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 16978e27ed..d8c59ab308 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -104,6 +104,8 @@ struct virtio_user_backend_ops {
int (*set_owner)(struct virtio_user_dev *dev);
int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
+ int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t *features);
+ int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t features);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index d204fa1eb0..5e89b1d72d 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -199,6 +199,62 @@ vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
return 0;
}
+static int
+vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_GET_PROTOCOL_FEATURES,
+ .flags = VHOST_USER_VERSION,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0)
+ goto err;
+
+ ret = vhost_user_read(dev->vhostfd, &msg);
+ if (ret < 0)
+ goto err;
+
+ if (msg.request != VHOST_USER_GET_PROTOCOL_FEATURES) {
+ PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
+ goto err;
+ }
+
+ if (msg.size != sizeof(*features)) {
+ PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
+ goto err;
+ }
+
+ *features = msg.payload.u64;
+
+ return 0;
+err:
+ PMD_DRV_LOG(ERR, "Failed to get backend protocol features");
+
+ return -1;
+}
+
+static int
+vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_PROTOCOL_FEATURES,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(features),
+ .payload.u64 = features,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set protocol features");
+ return -1;
+ }
+
+ return 0;
+}
+
struct walk_arg {
struct vhost_memory *vm;
int *fds;
@@ -313,8 +369,6 @@ const char * const vhost_msg_strings[] = {
[VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
[VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
[VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
- [VHOST_USER_GET_PROTOCOL_FEATURES] = "VHOST_USER_GET_PROTOCOL_FEATURES",
- [VHOST_USER_SET_PROTOCOL_FEATURES] = "VHOST_USER_SET_PROTOCOL_FEATURES",
[VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
[VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
};
@@ -352,8 +406,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
(!(dev->protocol_features &
(1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
return -ENOTSUP;
- /* Fallthrough */
- case VHOST_USER_GET_PROTOCOL_FEATURES:
need_reply = 1;
break;
@@ -366,7 +418,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
if (has_reply_ack)
msg.flags |= VHOST_USER_NEED_REPLY_MASK;
/* Fallthrough */
- case VHOST_USER_SET_PROTOCOL_FEATURES:
case VHOST_USER_SET_LOG_BASE:
msg.payload.u64 = *((__u64 *)arg);
msg.size = sizeof(m.payload.u64);
@@ -446,7 +497,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
switch (req) {
case VHOST_USER_GET_STATUS:
- case VHOST_USER_GET_PROTOCOL_FEATURES:
if (msg.size != sizeof(m.payload.u64)) {
PMD_DRV_LOG(ERR, "Received bad msg size");
return -1;
@@ -584,6 +634,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_owner = vhost_user_set_owner,
.get_features = vhost_user_get_features,
.set_features = vhost_user_set_features,
+ .get_protocol_features = vhost_user_get_protocol_features,
+ .set_protocol_features = vhost_user_set_protocol_features,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 0a85d058a8..6bb61b3e89 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -495,24 +495,17 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
- if (dev->device_features &
- (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->send_request(dev,
- VHOST_USER_GET_PROTOCOL_FEATURES,
- &protocol_features))
+ if (dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
+ if (dev->ops->get_protocol_features(dev, &protocol_features))
return -1;
dev->protocol_features &= protocol_features;
- if (dev->ops->send_request(dev,
- VHOST_USER_SET_PROTOCOL_FEATURES,
- &dev->protocol_features))
+ if (dev->ops->set_protocol_features(dev, dev->protocol_features))
return -1;
- if (!(dev->protocol_features &
- (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
- dev->unsupported_features |=
- (1ull << VIRTIO_NET_F_MQ);
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
+ dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
}
} else {
/* We just pretend vhost-user can support all these features.
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 4d2635c8aa..c63d010e16 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -93,23 +93,17 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
if (dev->device_features &
(1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->send_request(dev,
- VHOST_USER_GET_PROTOCOL_FEATURES,
- &protocol_features))
+ if (dev->ops->get_protocol_features(dev, &protocol_features))
return -1;
/* Offer VHOST_USER_PROTOCOL_F_STATUS */
- dev->protocol_features |=
- (1ULL << VHOST_USER_PROTOCOL_F_STATUS);
+ dev->protocol_features |= (1ULL << VHOST_USER_PROTOCOL_F_STATUS);
dev->protocol_features &= protocol_features;
- if (dev->ops->send_request(dev,
- VHOST_USER_SET_PROTOCOL_FEATURES,
- &dev->protocol_features))
+ if (dev->ops->set_protocol_features(dev, dev->protocol_features))
return -1;
- if (!(dev->protocol_features &
- (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 26/40] net/virtio: add Virtio-user protocol features ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 26/40] net/virtio: add Virtio-user protocol " Maxime Coquelin
@ 2021-01-06 11:55 ` Xia, Chenbo
0 siblings, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 11:55 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 26/40] net/virtio: add Virtio-user protocol features ops
>
> This patch introduce new callbacks for getting
s/introduce/introduces
Thanks,
Chenbo
> and setting Vhost-user protocol features.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (25 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 26/40] net/virtio: add Virtio-user protocol " Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 11:57 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops Maxime Coquelin
` (13 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch implements a dedicated callback for
preparing and sending memory table to the backends.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 1 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 60 +++++++--------
drivers/net/virtio/virtio_user/vhost_user.c | 73 ++++++++++++++-----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 +-
.../net/virtio/virtio_user/virtio_user_dev.c | 4 +-
5 files changed, 88 insertions(+), 58 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index d8c59ab308..0a582a6844 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -106,6 +106,7 @@ struct virtio_user_backend_ops {
int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t *features);
int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t features);
+ int (*set_memory_table)(struct virtio_user_dev *dev);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index f44df8ef1f..2d30f572b6 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -148,17 +148,6 @@ vhost_kernel_set_features(struct virtio_user_dev *dev, uint64_t features)
return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_FEATURES, &features);
}
-static uint64_t vhost_req_user_to_kernel[] = {
- [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
- [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
- [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
- [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
- [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
- [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
- [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
-};
-
static int
add_memseg_list(const struct rte_memseg_list *msl, void *arg)
{
@@ -193,16 +182,17 @@ add_memseg_list(const struct rte_memseg_list *msl, void *arg)
* have much more memory regions. Below function will treat each
* contiguous memory space reserved by DPDK as one region.
*/
-static struct vhost_memory_kernel *
-prepare_vhost_memory_kernel(void)
+static int
+vhost_kernel_set_memory_table(struct virtio_user_dev *dev)
{
struct vhost_memory_kernel *vm;
+ int ret;
vm = malloc(sizeof(struct vhost_memory_kernel) +
max_regions *
sizeof(struct vhost_memory_region));
if (!vm)
- return NULL;
+ goto err;
vm->nregions = 0;
vm->padding = 0;
@@ -211,14 +201,34 @@ prepare_vhost_memory_kernel(void)
* The memory lock has already been taken by memory subsystem
* or virtio_user_start_device().
*/
- if (rte_memseg_list_walk_thread_unsafe(add_memseg_list, vm) < 0) {
- free(vm);
- return NULL;
- }
+ ret = rte_memseg_list_walk_thread_unsafe(add_memseg_list, vm);
+ if (ret < 0)
+ goto err_free;
+
+ ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_MEM_TABLE, vm);
+ if (ret < 0)
+ goto err_free;
- return vm;
+ free(vm);
+
+ return 0;
+err_free:
+ free(vm);
+err:
+ PMD_DRV_LOG(ERR, "Failed to set memory table");
+ return -1;
}
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+};
+
static int
vhost_kernel_send_request(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -227,7 +237,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
int ret = -1;
unsigned int i;
uint64_t req_kernel;
- struct vhost_memory_kernel *vm = NULL;
int vhostfd;
unsigned int queue_sel;
@@ -235,13 +244,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
req_kernel = vhost_req_user_to_kernel[req];
- if (req_kernel == VHOST_SET_MEM_TABLE) {
- vm = prepare_vhost_memory_kernel();
- if (!vm)
- return -1;
- arg = (void *)vm;
- }
-
switch (req_kernel) {
case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
@@ -271,9 +273,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
ret = ioctl(vhostfd, req_kernel, arg);
}
- if (vm)
- free(vm);
-
if (ret < 0)
PMD_DRV_LOG(ERR, "%s failed: %s",
vhost_msg_strings[req], strerror(errno));
@@ -403,6 +402,7 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.set_owner = vhost_kernel_set_owner,
.get_features = vhost_kernel_get_features,
.set_features = vhost_kernel_set_features,
+ .set_memory_table = vhost_kernel_set_memory_table,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 5e89b1d72d..57f6337750 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -125,6 +125,29 @@ vhost_user_read(int fd, struct vhost_user_msg *msg)
return -1;
}
+static int
+vhost_user_check_reply_ack(struct virtio_user_dev *dev, struct vhost_user_msg *msg)
+{
+ enum vhost_user_request req = msg->request;
+ int ret;
+
+ if (!(msg->flags & VHOST_USER_NEED_REPLY_MASK))
+ return 0;
+
+ ret = vhost_user_read(dev->vhostfd, msg);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to read reply-ack");
+ return -1;
+ }
+
+ if (req != msg->request) {
+ PMD_DRV_LOG(ERR, "Unexpected reply-ack request type (%d)", msg->request);
+ return -1;
+ }
+
+ return msg->payload.u64 ? -1 : 0;
+}
+
static int
vhost_user_set_owner(struct virtio_user_dev *dev)
{
@@ -336,25 +359,47 @@ update_memory_region(const struct rte_memseg_list *msl __rte_unused,
}
static int
-prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
+vhost_user_set_memory_table(struct virtio_user_dev *dev)
{
struct walk_arg wa;
+ int fds[VHOST_MEMORY_MAX_NREGIONS];
+ int ret, fd_num;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_MEM_TABLE,
+ .flags = VHOST_USER_VERSION,
+ };
+
+ if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
+ msg.flags |= VHOST_USER_NEED_REPLY_MASK;
wa.region_nr = 0;
- wa.vm = &msg->payload.memory;
+ wa.vm = &msg.payload.memory;
wa.fds = fds;
/*
* The memory lock has already been taken by memory subsystem
* or virtio_user_start_device().
*/
- if (rte_memseg_walk_thread_unsafe(update_memory_region, &wa) < 0)
- return -1;
+ ret = rte_memseg_walk_thread_unsafe(update_memory_region, &wa);
+ if (ret < 0)
+ goto err;
- msg->payload.memory.nregions = wa.region_nr;
- msg->payload.memory.padding = 0;
+ fd_num = wa.region_nr;
+ msg.payload.memory.nregions = wa.region_nr;
+ msg.payload.memory.padding = 0;
- return 0;
+ msg.size = sizeof(msg.payload.memory.nregions);
+ msg.size += sizeof(msg.payload.memory.padding);
+ msg.size += fd_num * sizeof(struct vhost_memory_region);
+
+ ret = vhost_user_write(dev->vhostfd, &msg, fds, fd_num);
+ if (ret < 0)
+ goto err;
+
+ return vhost_user_check_reply_ack(dev, &msg);
+err:
+ PMD_DRV_LOG(ERR, "Failed to set memory table");
+ return -1;
}
static struct vhost_user_msg m;
@@ -367,7 +412,6 @@ const char * const vhost_msg_strings[] = {
[VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
[VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
[VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
[VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
[VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
[VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
@@ -426,18 +470,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
case VHOST_USER_RESET_OWNER:
break;
- case VHOST_USER_SET_MEM_TABLE:
- if (prepare_vhost_memory_user(&msg, fds) < 0)
- return -1;
- fd_num = msg.payload.memory.nregions;
- msg.size = sizeof(m.payload.memory.nregions);
- msg.size += sizeof(m.payload.memory.padding);
- msg.size += fd_num * sizeof(struct vhost_memory_region);
-
- if (has_reply_ack)
- msg.flags |= VHOST_USER_NEED_REPLY_MASK;
- break;
-
case VHOST_USER_SET_LOG_FD:
fds[fd_num++] = *((int *)arg);
break;
@@ -636,6 +668,7 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_features = vhost_user_set_features,
.get_protocol_features = vhost_user_get_protocol_features,
.set_protocol_features = vhost_user_set_protocol_features,
+ .set_memory_table = vhost_user_set_memory_table,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index c0a9b5b767..3059ec545d 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -19,7 +19,6 @@
#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
-#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, void *)
#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
@@ -44,7 +43,6 @@ static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
[VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
[VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
- [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
[VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
[VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
[VHOST_USER_SET_VRING_ENABLE] = VHOST_VDPA_SET_VRING_ENABLE,
@@ -202,7 +200,7 @@ vhost_vdpa_map(const struct rte_memseg_list *msl, const struct rte_memseg *ms,
}
static int
-vhost_vdpa_dma_map_all(struct virtio_user_dev *dev)
+vhost_vdpa_set_memory_table(struct virtio_user_dev *dev)
{
vhost_vdpa_dma_unmap(dev, NULL, 0, SIZE_MAX);
@@ -248,9 +246,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
req_vdpa = vhost_req_user_to_vdpa[req];
- if (req_vdpa == VHOST_SET_MEM_TABLE)
- return vhost_vdpa_dma_map_all(dev);
-
switch (req_vdpa) {
case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
@@ -331,6 +326,7 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.set_owner = vhost_vdpa_set_owner,
.get_features = vhost_vdpa_get_features,
.set_features = vhost_vdpa_set_features,
+ .set_memory_table = vhost_vdpa_set_memory_table,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 6bb61b3e89..ae976be158 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -177,7 +177,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
goto error;
/* Step 2: share memory regions */
- ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = dev->ops->set_memory_table(dev);
if (ret < 0)
goto error;
@@ -351,7 +351,7 @@ virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
dev->ops->enable_qp(dev, i, 0);
/* Step 2: update memory regions */
- dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
+ dev->ops->set_memory_table(dev);
/* Step 3: resume the active queues */
for (i = 0; i < dev->queue_pairs; i++)
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops Maxime Coquelin
@ 2021-01-06 11:57 ` Xia, Chenbo
2021-01-15 9:57 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 11:57 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 27/40] net/virtio: add Virtio-user memory tables ops
>
> This patch implements a dedicated callback for
> preparing and sending memory table to the backends.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
<snip>
>
> +static int
> +vhost_user_check_reply_ack(struct virtio_user_dev *dev, struct vhost_user_msg
> *msg)
> +{
> + enum vhost_user_request req = msg->request;
> + int ret;
> +
> + if (!(msg->flags & VHOST_USER_NEED_REPLY_MASK))
> + return 0;
> +
> + ret = vhost_user_read(dev->vhostfd, msg);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to read reply-ack");
> + return -1;
> + }
> +
> + if (req != msg->request) {
> + PMD_DRV_LOG(ERR, "Unexpected reply-ack request type (%d)", msg-
> >request);
> + return -1;
> + }
I think it's better to keep the size check: msg.size should equal sizeof(msg.payload.u64).
> +
> + return msg->payload.u64 ? -1 : 0;
I think it's better to add a log after checking payload's value. Looking back to
vhost_user_set_memory_table, there's no way for user or developer to know what has
failed (vhost_user_write fails or NACK). Maybe it's also better to add error log in
or outside vhost_user_write :)
Thanks,
Chenbo
> +}
> +
> static int
> vhost_user_set_owner(struct virtio_user_dev *dev)
> {
> @@ -336,25 +359,47 @@ update_memory_region(const struct rte_memseg_list *msl
> __rte_unused,
> }
>
> static int
> -prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
> +vhost_user_set_memory_table(struct virtio_user_dev *dev)
> {
> struct walk_arg wa;
> + int fds[VHOST_MEMORY_MAX_NREGIONS];
> + int ret, fd_num;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_SET_MEM_TABLE,
> + .flags = VHOST_USER_VERSION,
> + };
> +
> + if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> + msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>
> wa.region_nr = 0;
> - wa.vm = &msg->payload.memory;
> + wa.vm = &msg.payload.memory;
> wa.fds = fds;
>
> /*
> * The memory lock has already been taken by memory subsystem
> * or virtio_user_start_device().
> */
> - if (rte_memseg_walk_thread_unsafe(update_memory_region, &wa) < 0)
> - return -1;
> + ret = rte_memseg_walk_thread_unsafe(update_memory_region, &wa);
> + if (ret < 0)
> + goto err;
>
> - msg->payload.memory.nregions = wa.region_nr;
> - msg->payload.memory.padding = 0;
> + fd_num = wa.region_nr;
> + msg.payload.memory.nregions = wa.region_nr;
> + msg.payload.memory.padding = 0;
>
> - return 0;
> + msg.size = sizeof(msg.payload.memory.nregions);
> + msg.size += sizeof(msg.payload.memory.padding);
> + msg.size += fd_num * sizeof(struct vhost_memory_region);
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, fds, fd_num);
> + if (ret < 0)
> + goto err;
> +
> + return vhost_user_check_reply_ack(dev, &msg);
> +err:
> + PMD_DRV_LOG(ERR, "Failed to set memory table");
> + return -1;
> }
>
> static struct vhost_user_msg m;
> @@ -367,7 +412,6 @@ const char * const vhost_msg_strings[] = {
> [VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
> [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
> [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
> - [VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
> [VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
> [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
> [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
> @@ -426,18 +470,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> case VHOST_USER_RESET_OWNER:
> break;
>
> - case VHOST_USER_SET_MEM_TABLE:
> - if (prepare_vhost_memory_user(&msg, fds) < 0)
> - return -1;
> - fd_num = msg.payload.memory.nregions;
> - msg.size = sizeof(m.payload.memory.nregions);
> - msg.size += sizeof(m.payload.memory.padding);
> - msg.size += fd_num * sizeof(struct vhost_memory_region);
> -
> - if (has_reply_ack)
> - msg.flags |= VHOST_USER_NEED_REPLY_MASK;
> - break;
> -
> case VHOST_USER_SET_LOG_FD:
> fds[fd_num++] = *((int *)arg);
> break;
> @@ -636,6 +668,7 @@ struct virtio_user_backend_ops virtio_ops_user = {
> .set_features = vhost_user_set_features,
> .get_protocol_features = vhost_user_get_protocol_features,
> .set_protocol_features = vhost_user_set_protocol_features,
> + .set_memory_table = vhost_user_set_memory_table,
> .send_request = vhost_user_sock,
> .enable_qp = vhost_user_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index c0a9b5b767..3059ec545d 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -19,7 +19,6 @@
> #define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
> #define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
> #define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
> -#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, void *)
> #define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
> #define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
> #define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
> @@ -44,7 +43,6 @@ static uint64_t vhost_req_user_to_vdpa[] = {
> [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
> [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
> - [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
> [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
> [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
> [VHOST_USER_SET_VRING_ENABLE] = VHOST_VDPA_SET_VRING_ENABLE,
> @@ -202,7 +200,7 @@ vhost_vdpa_map(const struct rte_memseg_list *msl, const
> struct rte_memseg *ms,
> }
>
> static int
> -vhost_vdpa_dma_map_all(struct virtio_user_dev *dev)
> +vhost_vdpa_set_memory_table(struct virtio_user_dev *dev)
> {
> vhost_vdpa_dma_unmap(dev, NULL, 0, SIZE_MAX);
>
> @@ -248,9 +246,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
>
> req_vdpa = vhost_req_user_to_vdpa[req];
>
> - if (req_vdpa == VHOST_SET_MEM_TABLE)
> - return vhost_vdpa_dma_map_all(dev);
> -
> switch (req_vdpa) {
> case VHOST_SET_VRING_NUM:
> case VHOST_SET_VRING_ADDR:
> @@ -331,6 +326,7 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
> .set_owner = vhost_vdpa_set_owner,
> .get_features = vhost_vdpa_get_features,
> .set_features = vhost_vdpa_set_features,
> + .set_memory_table = vhost_vdpa_set_memory_table,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 6bb61b3e89..ae976be158 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -177,7 +177,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> goto error;
>
> /* Step 2: share memory regions */
> - ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
> + ret = dev->ops->set_memory_table(dev);
> if (ret < 0)
> goto error;
>
> @@ -351,7 +351,7 @@ virtio_user_mem_event_cb(enum rte_mem_event type
> __rte_unused,
> dev->ops->enable_qp(dev, i, 0);
>
> /* Step 2: update memory regions */
> - dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
> + dev->ops->set_memory_table(dev);
>
> /* Step 3: resume the active queues */
> for (i = 0; i < dev->queue_pairs; i++)
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops
2021-01-06 11:57 ` Xia, Chenbo
@ 2021-01-15 9:57 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 9:57 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 12:57 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 27/40] net/virtio: add Virtio-user memory tables ops
>>
>> This patch implements a dedicated callback for
>> preparing and sending memory table to the backends.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>
> <snip>
>
>>
>> +static int
>> +vhost_user_check_reply_ack(struct virtio_user_dev *dev, struct vhost_user_msg
>> *msg)
>> +{
>> + enum vhost_user_request req = msg->request;
>> + int ret;
>> +
>> + if (!(msg->flags & VHOST_USER_NEED_REPLY_MASK))
>> + return 0;
>> +
>> + ret = vhost_user_read(dev->vhostfd, msg);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to read reply-ack");
>> + return -1;
>> + }
>> +
>> + if (req != msg->request) {
>> + PMD_DRV_LOG(ERR, "Unexpected reply-ack request type (%d)", msg-
>>> request);
>> + return -1;
>> + }
>
> I think it's better to keep the size check: msg.size should equal sizeof(msg.payload.u64).
>
>> +
>> + return msg->payload.u64 ? -1 : 0;
>
> I think it's better to add a log after checking payload's value. Looking back to
> vhost_user_set_memory_table, there's no way for user or developer to know what has
> failed (vhost_user_write fails or NACK). Maybe it's also better to add error log in
> or outside vhost_user_write :)
Indeed, we are lacking logs here, I added both a log when slave replies
NACK and when sendmsg fails.
Thanks,
Maxime
> Thanks,
> Chenbo
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (26 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 27/40] net/virtio: add Virtio-user memory tables ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-05 21:24 ` David Marchand
` (2 more replies)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops Maxime Coquelin
` (12 subsequent siblings)
40 siblings, 3 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduces new callbacks for setting
and getting vring state.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 4 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 49 +++++++-
drivers/net/virtio/virtio_user/vhost_user.c | 114 +++++++++++++-----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 40 ++++--
.../net/virtio/virtio_user/virtio_user_dev.c | 9 +-
5 files changed, 168 insertions(+), 48 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 0a582a6844..1385c1563b 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -107,6 +107,10 @@ struct virtio_user_backend_ops {
int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t *features);
int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t features);
int (*set_memory_table)(struct virtio_user_dev *dev);
+ int (*set_vring_enable)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
+ int (*set_vring_num)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
+ int (*set_vring_base)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
+ int (*get_vring_base)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 2d30f572b6..2f1b4840ee 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -219,12 +219,49 @@ vhost_kernel_set_memory_table(struct virtio_user_dev *dev)
return -1;
}
+static int
+vhost_kernel_set_vring(struct virtio_user_dev *dev, uint64_t req, struct vhost_vring_state *state)
+{
+ int ret, fd;
+ uint32_t index = state->index;
+
+ /* Convert from queue index to queue-pair & offset */
+ fd = dev->vhostfds[state->index / 2];
+ state->index %= 2;
+
+ ret = vhost_kernel_ioctl(fd, req, state);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set vring (request %lu)", req);
+ return -1;
+ }
+
+ /* restore index back to queue index */
+ state->index = index;
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_vring_num(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_kernel_set_vring(dev, VHOST_SET_VRING_NUM, state);
+}
+
+static int
+vhost_kernel_set_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_kernel_set_vring(dev, VHOST_SET_VRING_BASE, state);
+}
+
+static int
+vhost_kernel_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_kernel_set_vring(dev, VHOST_GET_VRING_BASE, state);
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
[VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
- [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
- [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
- [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
[VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
[VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
};
@@ -245,10 +282,7 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
req_kernel = vhost_req_user_to_kernel[req];
switch (req_kernel) {
- case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
- case VHOST_SET_VRING_BASE:
- case VHOST_GET_VRING_BASE:
case VHOST_SET_VRING_KICK:
case VHOST_SET_VRING_CALL:
queue_sel = *(unsigned int *)arg;
@@ -403,6 +437,9 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.get_features = vhost_kernel_get_features,
.set_features = vhost_kernel_set_features,
.set_memory_table = vhost_kernel_set_memory_table,
+ .set_vring_num = vhost_kernel_set_vring_num,
+ .set_vring_base = vhost_kernel_set_vring_base,
+ .get_vring_base = vhost_kernel_get_vring_base,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 57f6337750..50a587fab4 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -402,17 +402,94 @@ vhost_user_set_memory_table(struct virtio_user_dev *dev)
return -1;
}
+static int
+vhost_user_set_vring(struct virtio_user_dev *dev, enum vhost_user_request req,
+ struct vhost_vring_state *state)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = req,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(*state),
+ .payload.state = *state,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set vring state (request %d)", req);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_user_set_vring_enable(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_user_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, state);
+}
+
+static int
+vhost_user_set_vring_num(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_user_set_vring(dev, VHOST_USER_SET_VRING_NUM, state);
+}
+
+static int
+vhost_user_set_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_user_set_vring(dev, VHOST_USER_SET_VRING_BASE, state);
+}
+
+static int
+vhost_user_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ int ret;
+ struct vhost_user_msg msg;
+ unsigned int index = state->index;
+
+ ret = vhost_user_set_vring(dev, VHOST_USER_GET_VRING_BASE, state);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to send request");
+ goto err;
+ }
+
+ ret = vhost_user_read(dev->vhostfd, &msg);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to read reply");
+ goto err;
+ }
+
+ if (msg.request != VHOST_USER_GET_VRING_BASE) {
+ PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
+ goto err;
+ }
+
+ if (msg.size != sizeof(*state)) {
+ PMD_DRV_LOG(ERR, "Unexpected payload size (%u)", msg.size);
+ goto err;
+ }
+
+ if (msg.payload.state.index != index) {
+ PMD_DRV_LOG(ERR, "Unexpected ring index (%u)", state->index);
+ goto err;
+ }
+
+ *state = msg.payload.state;
+
+ return 0;
+err:
+ PMD_DRV_LOG(ERR, "Failed to get vring base");
+ return -1;
+}
+
static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
[VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
[VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
[VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
[VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
[VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
};
@@ -474,19 +551,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
fds[fd_num++] = *((int *)arg);
break;
- case VHOST_USER_SET_VRING_NUM:
- case VHOST_USER_SET_VRING_BASE:
- case VHOST_USER_SET_VRING_ENABLE:
- memcpy(&msg.payload.state, arg, sizeof(msg.payload.state));
- msg.size = sizeof(m.payload.state);
- break;
-
- case VHOST_USER_GET_VRING_BASE:
- memcpy(&msg.payload.state, arg, sizeof(msg.payload.state));
- msg.size = sizeof(m.payload.state);
- need_reply = 1;
- break;
-
case VHOST_USER_SET_VRING_ADDR:
memcpy(&msg.payload.addr, arg, sizeof(msg.payload.addr));
msg.size = sizeof(m.payload.addr);
@@ -535,14 +599,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
}
*((__u64 *)arg) = msg.payload.u64;
break;
- case VHOST_USER_GET_VRING_BASE:
- if (msg.size != sizeof(m.payload.state)) {
- PMD_DRV_LOG(ERR, "Received bad msg size");
- return -1;
- }
- memcpy(arg, &msg.payload.state,
- sizeof(struct vhost_vring_state));
- break;
default:
/* Reply-ack handling */
if (msg.size != sizeof(m.payload.u64)) {
@@ -650,10 +706,10 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
for (i = 0; i < 2; ++i) {
struct vhost_vring_state state = {
.index = pair_idx * 2 + i,
- .num = enable,
+ .num = enable,
};
- if (vhost_user_sock(dev, VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_user_set_vring_enable(dev, &state))
return -1;
}
@@ -669,6 +725,10 @@ struct virtio_user_backend_ops virtio_ops_user = {
.get_protocol_features = vhost_user_get_protocol_features,
.set_protocol_features = vhost_user_set_protocol_features,
.set_memory_table = vhost_user_set_memory_table,
+ .set_vring_enable = vhost_user_set_vring_enable,
+ .set_vring_num = vhost_user_set_vring_num,
+ .set_vring_base = vhost_user_set_vring_base,
+ .get_vring_base = vhost_user_get_vring_base,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index 3059ec545d..c1b790ddc6 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -32,20 +32,15 @@
#define VHOST_VDPA_GET_DEVICE_ID _IOR(VHOST_VIRTIO, 0x70, __u32)
#define VHOST_VDPA_GET_STATUS _IOR(VHOST_VIRTIO, 0x71, __u8)
#define VHOST_VDPA_SET_STATUS _IOW(VHOST_VIRTIO, 0x72, __u8)
-#define VHOST_VDPA_SET_VRING_ENABLE _IOW(VHOST_VIRTIO, 0x75, \
- struct vhost_vring_state)
+#define VHOST_VDPA_SET_VRING_ENABLE _IOW(VHOST_VIRTIO, 0x75, struct vhost_vring_state)
static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
[VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
- [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
- [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
- [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
[VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
[VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
[VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
[VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
- [VHOST_USER_SET_VRING_ENABLE] = VHOST_VDPA_SET_VRING_ENABLE,
};
/* no alignment requirement */
@@ -219,6 +214,30 @@ vhost_vdpa_set_memory_table(struct virtio_user_dev *dev)
return rte_memseg_walk_thread_unsafe(vhost_vdpa_map, dev);
}
+static int
+vhost_vdpa_set_vring_enable(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_VRING_ENABLE, state);
+}
+
+static int
+vhost_vdpa_set_vring_num(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_NUM, state);
+}
+
+static int
+vhost_vdpa_set_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_BASE, state);
+}
+
+static int
+vhost_vdpa_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_VRING_BASE, state);
+}
+
/* with below features, vhost vdpa does not need to do the checksum and TSO,
* these info will be passed to virtio_user through virtio net header.
*/
@@ -247,10 +266,7 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
req_vdpa = vhost_req_user_to_vdpa[req];
switch (req_vdpa) {
- case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_ADDR:
- case VHOST_SET_VRING_BASE:
- case VHOST_GET_VRING_BASE:
case VHOST_SET_VRING_KICK:
case VHOST_SET_VRING_CALL:
PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
@@ -312,7 +328,7 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
.num = enable,
};
- if (vhost_vdpa_send_request(dev, VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_vdpa_set_vring_enable(dev, &state))
return -1;
}
@@ -327,6 +343,10 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.get_features = vhost_vdpa_get_features,
.set_features = vhost_vdpa_set_features,
.set_memory_table = vhost_vdpa_set_memory_table,
+ .set_vring_enable = vhost_vdpa_set_vring_enable,
+ .set_vring_num = vhost_vdpa_set_vring_num,
+ .set_vring_base = vhost_vdpa_set_vring_base,
+ .get_vring_base = vhost_vdpa_get_vring_base,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index ae976be158..496a48ee51 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -73,13 +73,13 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
+ dev->ops->set_vring_num(dev, &state);
state.index = queue_sel;
state.num = 0; /* no reservation */
if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
state.num |= (1 << 15);
- dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
+ dev->ops->set_vring_base(dev, &state);
dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
@@ -218,9 +218,8 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
/* Stop the backend. */
for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
state.index = i;
- if (dev->ops->send_request(dev, VHOST_USER_GET_VRING_BASE,
- &state) < 0) {
- PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u\n",
+ if (dev->ops->get_vring_base(dev, &state) < 0) {
+ PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
i);
error = -1;
goto out;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops Maxime Coquelin
@ 2021-01-05 21:24 ` David Marchand
2021-01-06 12:01 ` Xia, Chenbo
2021-01-06 12:03 ` Xia, Chenbo
2 siblings, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:24 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
Skipping to the build error, I did not review this patch yet.
On Sun, Dec 20, 2020 at 10:16 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -219,12 +219,49 @@ vhost_kernel_set_memory_table(struct virtio_user_dev *dev)
> return -1;
> }
>
> +static int
> +vhost_kernel_set_vring(struct virtio_user_dev *dev, uint64_t req, struct vhost_vring_state *state)
> +{
> + int ret, fd;
> + uint32_t index = state->index;
> +
> + /* Convert from queue index to queue-pair & offset */
> + fd = dev->vhostfds[state->index / 2];
> + state->index %= 2;
> +
> + ret = vhost_kernel_ioctl(fd, req, state);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring (request %lu)", req);
Breaks 32bits build.
> + return -1;
> + }
> +
> + /* restore index back to queue index */
> + state->index = index;
> +
> + return 0;
> +}
> +
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops Maxime Coquelin
2021-01-05 21:24 ` David Marchand
@ 2021-01-06 12:01 ` Xia, Chenbo
2021-01-15 10:12 ` Maxime Coquelin
2021-01-06 12:03 ` Xia, Chenbo
2 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 12:01 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
>
> This patch introduces new callbacks for setting
> and getting vring state.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 4 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 49 +++++++-
> drivers/net/virtio/virtio_user/vhost_user.c | 114 +++++++++++++-----
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 40 ++++--
> .../net/virtio/virtio_user/virtio_user_dev.c | 9 +-
> 5 files changed, 168 insertions(+), 48 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index 0a582a6844..1385c1563b 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -107,6 +107,10 @@ struct virtio_user_backend_ops {
> int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t
> *features);
> int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t
> features);
> int (*set_memory_table)(struct virtio_user_dev *dev);
> + int (*set_vring_enable)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> + int (*set_vring_num)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> + int (*set_vring_base)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> + int (*get_vring_base)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 2d30f572b6..2f1b4840ee 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -219,12 +219,49 @@ vhost_kernel_set_memory_table(struct virtio_user_dev
> *dev)
> return -1;
> }
>
> +static int
> +vhost_kernel_set_vring(struct virtio_user_dev *dev, uint64_t req, struct
> vhost_vring_state *state)
> +{
> + int ret, fd;
> + uint32_t index = state->index;
Better use 'unsigned int index' here? It can hardly cause problem but I think it's better
to use the type in struct vhost_vring_state.
> +
> + /* Convert from queue index to queue-pair & offset */
> + fd = dev->vhostfds[state->index / 2];
> + state->index %= 2;
> +
> + ret = vhost_kernel_ioctl(fd, req, state);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring (request %lu)", req);
Seems David has also noticed here: better use PRIu64 here instead of %lu ?
Thanks,
Chenbo
> + return -1;
> + }
> +
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2021-01-06 12:01 ` Xia, Chenbo
@ 2021-01-15 10:12 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:12 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 1:01 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
>>
>> This patch introduces new callbacks for setting
>> and getting vring state.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 4 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 49 +++++++-
>> drivers/net/virtio/virtio_user/vhost_user.c | 114 +++++++++++++-----
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 40 ++++--
>> .../net/virtio/virtio_user/virtio_user_dev.c | 9 +-
>> 5 files changed, 168 insertions(+), 48 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index 0a582a6844..1385c1563b 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -107,6 +107,10 @@ struct virtio_user_backend_ops {
>> int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t
>> *features);
>> int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t
>> features);
>> int (*set_memory_table)(struct virtio_user_dev *dev);
>> + int (*set_vring_enable)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> + int (*set_vring_num)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> + int (*set_vring_base)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> + int (*get_vring_base)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> int (*send_request)(struct virtio_user_dev *dev,
>> enum vhost_user_request req,
>> void *arg);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 2d30f572b6..2f1b4840ee 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -219,12 +219,49 @@ vhost_kernel_set_memory_table(struct virtio_user_dev
>> *dev)
>> return -1;
>> }
>>
>> +static int
>> +vhost_kernel_set_vring(struct virtio_user_dev *dev, uint64_t req, struct
>> vhost_vring_state *state)
>> +{
>> + int ret, fd;
>> + uint32_t index = state->index;
>
> Better use 'unsigned int index' here? It can hardly cause problem but I think it's better
> to use the type in struct vhost_vring_state.
Agree.
>> +
>> + /* Convert from queue index to queue-pair & offset */
>> + fd = dev->vhostfds[state->index / 2];
>> + state->index %= 2;
>> +
>> + ret = vhost_kernel_ioctl(fd, req, state);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to set vring (request %lu)", req);
>
> Seems David has also noticed here: better use PRIu64 here instead of %lu ?
Yes, already fixed.
Thanks,
Maxime
> Thanks,
> Chenbo
>
>> + return -1;
>> + }
>> +
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops Maxime Coquelin
2021-01-05 21:24 ` David Marchand
2021-01-06 12:01 ` Xia, Chenbo
@ 2021-01-06 12:03 ` Xia, Chenbo
2021-01-15 10:15 ` Maxime Coquelin
2 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 12:03 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
>
> This patch introduces new callbacks for setting
> and getting vring state.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
<snip>
>
> @@ -327,6 +343,10 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
> .get_features = vhost_vdpa_get_features,
> .set_features = vhost_vdpa_set_features,
> .set_memory_table = vhost_vdpa_set_memory_table,
> + .set_vring_enable = vhost_vdpa_set_vring_enable,
Sorry, miss one comment in last email...
Do we still need to keep set_vring_enable in struct virtio_user_backend_ops? Because
as I notice, it's called only in another callback (enable_qp).
Thanks,
Chenbo
> + .set_vring_num = vhost_vdpa_set_vring_num,
> + .set_vring_base = vhost_vdpa_set_vring_base,
> + .get_vring_base = vhost_vdpa_get_vring_base,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index ae976be158..496a48ee51 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -73,13 +73,13 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
>
> state.index = queue_sel;
> state.num = vring->num;
> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
> + dev->ops->set_vring_num(dev, &state);
>
> state.index = queue_sel;
> state.num = 0; /* no reservation */
> if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
> state.num |= (1 << 15);
> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
> + dev->ops->set_vring_base(dev, &state);
>
> dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
>
> @@ -218,9 +218,8 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
> /* Stop the backend. */
> for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
> state.index = i;
> - if (dev->ops->send_request(dev, VHOST_USER_GET_VRING_BASE,
> - &state) < 0) {
> - PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u\n",
> + if (dev->ops->get_vring_base(dev, &state) < 0) {
> + PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
> i);
> error = -1;
> goto out;
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
2021-01-06 12:03 ` Xia, Chenbo
@ 2021-01-15 10:15 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:15 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 1:03 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 28/40] net/virtio: add Virtio-user vring setting ops
>>
>> This patch introduces new callbacks for setting
>> and getting vring state.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
> <snip>
>
>>
>> @@ -327,6 +343,10 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
>> .get_features = vhost_vdpa_get_features,
>> .set_features = vhost_vdpa_set_features,
>> .set_memory_table = vhost_vdpa_set_memory_table,
>> + .set_vring_enable = vhost_vdpa_set_vring_enable,
>
> Sorry, miss one comment in last email...
>
> Do we still need to keep set_vring_enable in struct virtio_user_backend_ops? Because
> as I notice, it's called only in another callback (enable_qp).
Right, it can be dropped now. I'll remove it.
Thanks,
Maxime
> Thanks,
> Chenbo
>
>> + .set_vring_num = vhost_vdpa_set_vring_num,
>> + .set_vring_base = vhost_vdpa_set_vring_base,
>> + .get_vring_base = vhost_vdpa_get_vring_base,
>> .send_request = vhost_vdpa_send_request,
>> .enable_qp = vhost_vdpa_enable_queue_pair,
>> .dma_map = vhost_vdpa_dma_map,
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index ae976be158..496a48ee51 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -73,13 +73,13 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
>> uint32_t queue_sel)
>>
>> state.index = queue_sel;
>> state.num = vring->num;
>> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
>> + dev->ops->set_vring_num(dev, &state);
>>
>> state.index = queue_sel;
>> state.num = 0; /* no reservation */
>> if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
>> state.num |= (1 << 15);
>> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
>> + dev->ops->set_vring_base(dev, &state);
>>
>> dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
>>
>> @@ -218,9 +218,8 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
>> /* Stop the backend. */
>> for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
>> state.index = i;
>> - if (dev->ops->send_request(dev, VHOST_USER_GET_VRING_BASE,
>> - &state) < 0) {
>> - PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u\n",
>> + if (dev->ops->get_vring_base(dev, &state) < 0) {
>> + PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
>> i);
>> error = -1;
>> goto out;
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (27 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 28/40] net/virtio: add Virtio-user vring setting ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-05 21:24 ` David Marchand
2021-01-06 12:04 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops Maxime Coquelin
` (11 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduces new callbacks for setting
vring files (kick and call).
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 41 +++++++++++++++--
drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++++--
drivers/net/virtio/virtio_user/vhost_vdpa.c | 18 ++++++--
.../net/virtio/virtio_user/virtio_user_dev.c | 4 +-
5 files changed, 97 insertions(+), 14 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 1385c1563b..b296ee215d 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -111,6 +111,8 @@ struct virtio_user_backend_ops {
int (*set_vring_num)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
int (*set_vring_base)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
int (*get_vring_base)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
+ int (*set_vring_call)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
+ int (*set_vring_kick)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 2f1b4840ee..1805aee7f7 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -259,11 +259,44 @@ vhost_kernel_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_stat
return vhost_kernel_set_vring(dev, VHOST_GET_VRING_BASE, state);
}
+static int
+vhost_kernel_set_vring_file(struct virtio_user_dev *dev, uint64_t req,
+ struct vhost_vring_file *file)
+{
+ int ret, fd;
+ uint32_t index = file->index;
+
+ /* Convert from queue index to queue-pair & offset */
+ fd = dev->vhostfds[file->index / 2];
+ file->index %= 2;
+
+ ret = vhost_kernel_ioctl(fd, req, file);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set vring file (request %lu)", req);
+ return -1;
+ }
+
+ /* restore index back to queue index */
+ file->index = index;
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_CALL, file);
+}
+
+static int
+vhost_kernel_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
[VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
- [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
};
static int
@@ -283,8 +316,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
switch (req_kernel) {
case VHOST_SET_VRING_ADDR:
- case VHOST_SET_VRING_KICK:
- case VHOST_SET_VRING_CALL:
queue_sel = *(unsigned int *)arg;
vhostfd = dev->vhostfds[queue_sel / 2];
*(unsigned int *)arg = queue_sel % 2;
@@ -440,6 +471,8 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.set_vring_num = vhost_kernel_set_vring_num,
.set_vring_base = vhost_kernel_set_vring_base,
.get_vring_base = vhost_kernel_get_vring_base,
+ .set_vring_call = vhost_kernel_set_vring_call,
+ .set_vring_kick = vhost_kernel_set_vring_kick,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 50a587fab4..509a96dfbc 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -483,13 +483,51 @@ vhost_user_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state
return -1;
}
+static int
+vhost_user_set_vring_file(struct virtio_user_dev *dev, enum vhost_user_request req,
+ struct vhost_vring_file *file)
+{
+ int ret;
+ int fd = file->fd;
+ int num_fd = 0;
+ struct vhost_user_msg msg = {
+ .request = req,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(msg.payload.u64),
+ .payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK,
+ };
+
+ if (fd >= 0)
+ num_fd++;
+ else
+ msg.payload.u64 |= VHOST_USER_VRING_NOFD_MASK;
+
+ ret = vhost_user_write(dev->vhostfd, &msg, &fd, num_fd);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set vring file (request %d)", req);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_user_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
+}
+
+static int
+vhost_user_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
+}
+
static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
[VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
[VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
[VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
};
@@ -556,8 +594,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
msg.size = sizeof(m.payload.addr);
break;
- case VHOST_USER_SET_VRING_KICK:
- case VHOST_USER_SET_VRING_CALL:
case VHOST_USER_SET_VRING_ERR:
file = arg;
msg.payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
@@ -729,6 +765,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_vring_num = vhost_user_set_vring_num,
.set_vring_base = vhost_user_set_vring_base,
.get_vring_base = vhost_user_get_vring_base,
+ .set_vring_call = vhost_user_set_vring_call,
+ .set_vring_kick = vhost_user_set_vring_kick,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index c1b790ddc6..48fe3f23e8 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -36,9 +36,7 @@
static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
[VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
- [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
[VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
[VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
};
@@ -238,6 +236,18 @@ vhost_vdpa_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state
return vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_VRING_BASE, state);
}
+static int
+vhost_vdpa_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_CALL, file);
+}
+
+static int
+vhost_vdpa_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *file)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_KICK, file);
+}
+
/* with below features, vhost vdpa does not need to do the checksum and TSO,
* these info will be passed to virtio_user through virtio net header.
*/
@@ -267,8 +277,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
switch (req_vdpa) {
case VHOST_SET_VRING_ADDR:
- case VHOST_SET_VRING_KICK:
- case VHOST_SET_VRING_CALL:
PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
dev->vhostfd, *(unsigned int *)arg);
break;
@@ -347,6 +355,8 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.set_vring_num = vhost_vdpa_set_vring_num,
.set_vring_base = vhost_vdpa_set_vring_base,
.get_vring_base = vhost_vdpa_get_vring_base,
+ .set_vring_call = vhost_vdpa_set_vring_call,
+ .set_vring_kick = vhost_vdpa_set_vring_kick,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 496a48ee51..e4975838ea 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -40,7 +40,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
file.index = queue_sel;
file.fd = dev->callfds[queue_sel];
- dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
+ dev->ops->set_vring_call(dev, &file);
return 0;
}
@@ -89,7 +89,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
*/
file.index = queue_sel;
file.fd = dev->kickfds[queue_sel];
- dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
+ dev->ops->set_vring_kick(dev, &file);
return 0;
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops Maxime Coquelin
@ 2021-01-05 21:24 ` David Marchand
2021-01-06 12:04 ` Xia, Chenbo
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:24 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:16 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 2f1b4840ee..1805aee7f7 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -259,11 +259,44 @@ vhost_kernel_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_stat
> return vhost_kernel_set_vring(dev, VHOST_GET_VRING_BASE, state);
> }
>
> +static int
> +vhost_kernel_set_vring_file(struct virtio_user_dev *dev, uint64_t req,
> + struct vhost_vring_file *file)
> +{
> + int ret, fd;
> + uint32_t index = file->index;
> +
> + /* Convert from queue index to queue-pair & offset */
> + fd = dev->vhostfds[file->index / 2];
> + file->index %= 2;
> +
> + ret = vhost_kernel_ioctl(fd, req, file);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring file (request %lu)", req);
Breaks 32bits build.
> + return -1;
> + }
> +
> + /* restore index back to queue index */
> + file->index = index;
> +
> + return 0;
> +}
> +
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops Maxime Coquelin
2021-01-05 21:24 ` David Marchand
@ 2021-01-06 12:04 ` Xia, Chenbo
2021-01-15 10:17 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 12:04 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 29/40] net/virtio: add Virtio-user vring file ops
>
> This patch introduces new callbacks for setting
> vring files (kick and call).
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 2 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 41 +++++++++++++++--
> drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++++--
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 18 ++++++--
> .../net/virtio/virtio_user/virtio_user_dev.c | 4 +-
> 5 files changed, 97 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index 1385c1563b..b296ee215d 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -111,6 +111,8 @@ struct virtio_user_backend_ops {
> int (*set_vring_num)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> int (*set_vring_base)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> int (*get_vring_base)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> + int (*set_vring_call)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> + int (*set_vring_kick)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 2f1b4840ee..1805aee7f7 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -259,11 +259,44 @@ vhost_kernel_get_vring_base(struct virtio_user_dev *dev,
> struct vhost_vring_stat
> return vhost_kernel_set_vring(dev, VHOST_GET_VRING_BASE, state);
> }
>
> +static int
> +vhost_kernel_set_vring_file(struct virtio_user_dev *dev, uint64_t req,
> + struct vhost_vring_file *file)
> +{
> + int ret, fd;
> + uint32_t index = file->index;
Better use 'unsigned int index' here? It can hardly cause problem but I think it's
better to use the type in struct vhost_vring_file.
> +
> + /* Convert from queue index to queue-pair & offset */
> + fd = dev->vhostfds[file->index / 2];
> + file->index %= 2;
> +
> + ret = vhost_kernel_ioctl(fd, req, file);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring file (request %lu)", req);
Seems David has also noticed: better use PRIu64 here instead of %lu ?
Thanks,
Chenbo
> + return -1;
> + }
> +
> + /* restore index back to queue index */
> + file->index = index;
> +
> + return 0;
> +}
> +
> +static int
> +vhost_kernel_set_vring_kick(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_CALL, file);
> +}
> +
> +static int
> +vhost_kernel_set_vring_call(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
> +}
> +
> static uint64_t vhost_req_user_to_kernel[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
> [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> - [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
> };
>
> static int
> @@ -283,8 +316,6 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
>
> switch (req_kernel) {
> case VHOST_SET_VRING_ADDR:
> - case VHOST_SET_VRING_KICK:
> - case VHOST_SET_VRING_CALL:
> queue_sel = *(unsigned int *)arg;
> vhostfd = dev->vhostfds[queue_sel / 2];
> *(unsigned int *)arg = queue_sel % 2;
> @@ -440,6 +471,8 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
> .set_vring_num = vhost_kernel_set_vring_num,
> .set_vring_base = vhost_kernel_set_vring_base,
> .get_vring_base = vhost_kernel_get_vring_base,
> + .set_vring_call = vhost_kernel_set_vring_call,
> + .set_vring_kick = vhost_kernel_set_vring_kick,
> .send_request = vhost_kernel_send_request,
> .enable_qp = vhost_kernel_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
> b/drivers/net/virtio/virtio_user/vhost_user.c
> index 50a587fab4..509a96dfbc 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -483,13 +483,51 @@ vhost_user_get_vring_base(struct virtio_user_dev *dev,
> struct vhost_vring_state
> return -1;
> }
>
> +static int
> +vhost_user_set_vring_file(struct virtio_user_dev *dev, enum
> vhost_user_request req,
> + struct vhost_vring_file *file)
> +{
> + int ret;
> + int fd = file->fd;
> + int num_fd = 0;
> + struct vhost_user_msg msg = {
> + .request = req,
> + .flags = VHOST_USER_VERSION,
> + .size = sizeof(msg.payload.u64),
> + .payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK,
> + };
> +
> + if (fd >= 0)
> + num_fd++;
> + else
> + msg.payload.u64 |= VHOST_USER_VRING_NOFD_MASK;
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, &fd, num_fd);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring file (request %d)", req);
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +vhost_user_set_vring_call(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
> +}
> +
> +static int
> +vhost_user_set_vring_kick(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
> +}
> +
> static struct vhost_user_msg m;
>
> const char * const vhost_msg_strings[] = {
> [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
> - [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
> [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
> - [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
> [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
> [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
> };
> @@ -556,8 +594,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> msg.size = sizeof(m.payload.addr);
> break;
>
> - case VHOST_USER_SET_VRING_KICK:
> - case VHOST_USER_SET_VRING_CALL:
> case VHOST_USER_SET_VRING_ERR:
> file = arg;
> msg.payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
> @@ -729,6 +765,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
> .set_vring_num = vhost_user_set_vring_num,
> .set_vring_base = vhost_user_set_vring_base,
> .get_vring_base = vhost_user_get_vring_base,
> + .set_vring_call = vhost_user_set_vring_call,
> + .set_vring_kick = vhost_user_set_vring_kick,
> .send_request = vhost_user_sock,
> .enable_qp = vhost_user_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index c1b790ddc6..48fe3f23e8 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -36,9 +36,7 @@
>
> static uint64_t vhost_req_user_to_vdpa[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
> [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> - [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
> [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
> [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
> };
> @@ -238,6 +236,18 @@ vhost_vdpa_get_vring_base(struct virtio_user_dev *dev,
> struct vhost_vring_state
> return vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_VRING_BASE, state);
> }
>
> +static int
> +vhost_vdpa_set_vring_call(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_CALL, file);
> +}
> +
> +static int
> +vhost_vdpa_set_vring_kick(struct virtio_user_dev *dev, struct
> vhost_vring_file *file)
> +{
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_KICK, file);
> +}
> +
> /* with below features, vhost vdpa does not need to do the checksum and TSO,
> * these info will be passed to virtio_user through virtio net header.
> */
> @@ -267,8 +277,6 @@ vhost_vdpa_send_request(struct virtio_user_dev *dev,
>
> switch (req_vdpa) {
> case VHOST_SET_VRING_ADDR:
> - case VHOST_SET_VRING_KICK:
> - case VHOST_SET_VRING_CALL:
> PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
> dev->vhostfd, *(unsigned int *)arg);
> break;
> @@ -347,6 +355,8 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
> .set_vring_num = vhost_vdpa_set_vring_num,
> .set_vring_base = vhost_vdpa_set_vring_base,
> .get_vring_base = vhost_vdpa_get_vring_base,
> + .set_vring_call = vhost_vdpa_set_vring_call,
> + .set_vring_kick = vhost_vdpa_set_vring_kick,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 496a48ee51..e4975838ea 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -40,7 +40,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
>
> file.index = queue_sel;
> file.fd = dev->callfds[queue_sel];
> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
> + dev->ops->set_vring_call(dev, &file);
>
> return 0;
> }
> @@ -89,7 +89,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t
> queue_sel)
> */
> file.index = queue_sel;
> file.fd = dev->kickfds[queue_sel];
> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
> + dev->ops->set_vring_kick(dev, &file);
>
> return 0;
> }
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops
2021-01-06 12:04 ` Xia, Chenbo
@ 2021-01-15 10:17 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:17 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 1:04 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 29/40] net/virtio: add Virtio-user vring file ops
>>
>> This patch introduces new callbacks for setting
>> vring files (kick and call).
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 41 +++++++++++++++--
>> drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++++--
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 18 ++++++--
>> .../net/virtio/virtio_user/virtio_user_dev.c | 4 +-
>> 5 files changed, 97 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index 1385c1563b..b296ee215d 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -111,6 +111,8 @@ struct virtio_user_backend_ops {
>> int (*set_vring_num)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> int (*set_vring_base)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> int (*get_vring_base)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> + int (*set_vring_call)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> + int (*set_vring_kick)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> int (*send_request)(struct virtio_user_dev *dev,
>> enum vhost_user_request req,
>> void *arg);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 2f1b4840ee..1805aee7f7 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -259,11 +259,44 @@ vhost_kernel_get_vring_base(struct virtio_user_dev *dev,
>> struct vhost_vring_stat
>> return vhost_kernel_set_vring(dev, VHOST_GET_VRING_BASE, state);
>> }
>>
>> +static int
>> +vhost_kernel_set_vring_file(struct virtio_user_dev *dev, uint64_t req,
>> + struct vhost_vring_file *file)
>> +{
>> + int ret, fd;
>> + uint32_t index = file->index;
>
> Better use 'unsigned int index' here? It can hardly cause problem but I think it's
> better to use the type in struct vhost_vring_file.
Fixed.
>> +
>> + /* Convert from queue index to queue-pair & offset */
>> + fd = dev->vhostfds[file->index / 2];
>> + file->index %= 2;
>> +
>> + ret = vhost_kernel_ioctl(fd, req, file);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to set vring file (request %lu)", req);
>
> Seems David has also noticed: better use PRIu64 here instead of %lu ?
Yes.
Thanks,
Maxime
> Thanks,
> Chenbo
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (28 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 29/40] net/virtio: add Virtio-user vring file ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 12:06 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops Maxime Coquelin
` (10 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduces a new callback for setting
vrings addresses.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 1 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 32 +++++++++++++------
drivers/net/virtio/virtio_user/vhost_user.c | 29 +++++++++++++----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++-
.../net/virtio/virtio_user/virtio_user_dev.c | 2 +-
5 files changed, 55 insertions(+), 17 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index b296ee215d..956eb58728 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -113,6 +113,7 @@ struct virtio_user_backend_ops {
int (*get_vring_base)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
int (*set_vring_call)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
int (*set_vring_kick)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
+ int (*set_vring_addr)(struct virtio_user_dev *dev, struct vhost_vring_addr *addr);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 1805aee7f7..8cd86b72c6 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -294,9 +294,30 @@ vhost_kernel_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file
return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
}
+static int
+vhost_kernel_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *addr)
+{
+ int ret, fd;
+ uint32_t index = addr->index;
+
+ /* Convert from queue index to queue-pair & offset */
+ fd = dev->vhostfds[addr->index / 2];
+ addr->index %= 2;
+
+ ret = vhost_kernel_ioctl(fd, VHOST_SET_VRING_ADDR, addr);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to set vring address");
+ return -1;
+ }
+
+ /* restore index back to queue index */
+ addr->index = index;
+
+ return 0;
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
};
static int
@@ -308,20 +329,12 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
unsigned int i;
uint64_t req_kernel;
int vhostfd;
- unsigned int queue_sel;
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
req_kernel = vhost_req_user_to_kernel[req];
switch (req_kernel) {
- case VHOST_SET_VRING_ADDR:
- queue_sel = *(unsigned int *)arg;
- vhostfd = dev->vhostfds[queue_sel / 2];
- *(unsigned int *)arg = queue_sel % 2;
- PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
- vhostfd, *(unsigned int *)arg);
- break;
default:
vhostfd = -1;
}
@@ -473,6 +486,7 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.get_vring_base = vhost_kernel_get_vring_base,
.set_vring_call = vhost_kernel_set_vring_call,
.set_vring_kick = vhost_kernel_set_vring_kick,
+ .set_vring_addr = vhost_kernel_set_vring_addr,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 509a96dfbc..59cf366683 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -523,11 +523,32 @@ vhost_user_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *
return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
}
+
+static int
+vhost_user_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *addr)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_VRING_ADDR,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(*addr),
+ .payload.addr = *addr,
+ };
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to send vring addresses");
+ return -1;
+ }
+
+ return 0;
+}
+
+
static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
[VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
[VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
};
@@ -589,11 +610,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
fds[fd_num++] = *((int *)arg);
break;
- case VHOST_USER_SET_VRING_ADDR:
- memcpy(&msg.payload.addr, arg, sizeof(msg.payload.addr));
- msg.size = sizeof(m.payload.addr);
- break;
-
case VHOST_USER_SET_VRING_ERR:
file = arg;
msg.payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
@@ -767,6 +783,7 @@ struct virtio_user_backend_ops virtio_ops_user = {
.get_vring_base = vhost_user_get_vring_base,
.set_vring_call = vhost_user_set_vring_call,
.set_vring_kick = vhost_user_set_vring_kick,
+ .set_vring_addr = vhost_user_set_vring_addr,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index 48fe3f23e8..e09e7c9fb8 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -36,7 +36,6 @@
static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
[VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
[VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
};
@@ -248,6 +247,12 @@ vhost_vdpa_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *
return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_KICK, file);
}
+static int
+vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *addr)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
+}
+
/* with below features, vhost vdpa does not need to do the checksum and TSO,
* these info will be passed to virtio_user through virtio net header.
*/
@@ -357,6 +362,7 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.get_vring_base = vhost_vdpa_get_vring_base,
.set_vring_call = vhost_vdpa_set_vring_call,
.set_vring_kick = vhost_vdpa_set_vring_kick,
+ .set_vring_addr = vhost_vdpa_set_vring_addr,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e4975838ea..48ca7a2548 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -81,7 +81,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.num |= (1 << 15);
dev->ops->set_vring_base(dev, &state);
- dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
+ dev->ops->set_vring_addr(dev, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops Maxime Coquelin
@ 2021-01-06 12:06 ` Xia, Chenbo
2021-01-15 10:19 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 12:06 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 30/40] net/virtio: add Virtio-user vring address ops
>
> This patch introduces a new callback for setting
> vrings addresses.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 1 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 32 +++++++++++++------
> drivers/net/virtio/virtio_user/vhost_user.c | 29 +++++++++++++----
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++-
> .../net/virtio/virtio_user/virtio_user_dev.c | 2 +-
> 5 files changed, 55 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index b296ee215d..956eb58728 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -113,6 +113,7 @@ struct virtio_user_backend_ops {
> int (*get_vring_base)(struct virtio_user_dev *dev, struct
> vhost_vring_state *state);
> int (*set_vring_call)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> int (*set_vring_kick)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> + int (*set_vring_addr)(struct virtio_user_dev *dev, struct
> vhost_vring_addr *addr);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 1805aee7f7..8cd86b72c6 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -294,9 +294,30 @@ vhost_kernel_set_vring_call(struct virtio_user_dev *dev,
> struct vhost_vring_file
> return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
> }
>
> +static int
> +vhost_kernel_set_vring_addr(struct virtio_user_dev *dev, struct
> vhost_vring_addr *addr)
> +{
> + int ret, fd;
> + uint32_t index = addr->index;
Better use 'unsigned int index' here? It can hardly cause problem but I think
it's better to use the type in struct vhost_vring_addr.
Thanks,
Chenbo
> +
> + /* Convert from queue index to queue-pair & offset */
> + fd = dev->vhostfds[addr->index / 2];
> + addr->index %= 2;
> +
> + ret = vhost_kernel_ioctl(fd, VHOST_SET_VRING_ADDR, addr);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to set vring address");
> + return -1;
> + }
> +
> + /* restore index back to queue index */
> + addr->index = index;
> +
> + return 0;
> +}
> +
> static uint64_t vhost_req_user_to_kernel[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> };
>
> static int
> @@ -308,20 +329,12 @@ vhost_kernel_send_request(struct virtio_user_dev *dev,
> unsigned int i;
> uint64_t req_kernel;
> int vhostfd;
> - unsigned int queue_sel;
>
> PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
>
> req_kernel = vhost_req_user_to_kernel[req];
>
> switch (req_kernel) {
> - case VHOST_SET_VRING_ADDR:
> - queue_sel = *(unsigned int *)arg;
> - vhostfd = dev->vhostfds[queue_sel / 2];
> - *(unsigned int *)arg = queue_sel % 2;
> - PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
> - vhostfd, *(unsigned int *)arg);
> - break;
> default:
> vhostfd = -1;
> }
> @@ -473,6 +486,7 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
> .get_vring_base = vhost_kernel_get_vring_base,
> .set_vring_call = vhost_kernel_set_vring_call,
> .set_vring_kick = vhost_kernel_set_vring_kick,
> + .set_vring_addr = vhost_kernel_set_vring_addr,
> .send_request = vhost_kernel_send_request,
> .enable_qp = vhost_kernel_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
> b/drivers/net/virtio/virtio_user/vhost_user.c
> index 509a96dfbc..59cf366683 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -523,11 +523,32 @@ vhost_user_set_vring_kick(struct virtio_user_dev *dev,
> struct vhost_vring_file *
> return vhost_user_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
> }
>
> +
> +static int
> +vhost_user_set_vring_addr(struct virtio_user_dev *dev, struct
> vhost_vring_addr *addr)
> +{
> + int ret;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_SET_VRING_ADDR,
> + .flags = VHOST_USER_VERSION,
> + .size = sizeof(*addr),
> + .payload.addr = *addr,
> + };
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to send vring addresses");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +
> static struct vhost_user_msg m;
>
> const char * const vhost_msg_strings[] = {
> [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
> - [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
> [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
> [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
> };
> @@ -589,11 +610,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> fds[fd_num++] = *((int *)arg);
> break;
>
> - case VHOST_USER_SET_VRING_ADDR:
> - memcpy(&msg.payload.addr, arg, sizeof(msg.payload.addr));
> - msg.size = sizeof(m.payload.addr);
> - break;
> -
> case VHOST_USER_SET_VRING_ERR:
> file = arg;
> msg.payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
> @@ -767,6 +783,7 @@ struct virtio_user_backend_ops virtio_ops_user = {
> .get_vring_base = vhost_user_get_vring_base,
> .set_vring_call = vhost_user_set_vring_call,
> .set_vring_kick = vhost_user_set_vring_kick,
> + .set_vring_addr = vhost_user_set_vring_addr,
> .send_request = vhost_user_sock,
> .enable_qp = vhost_user_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index 48fe3f23e8..e09e7c9fb8 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -36,7 +36,6 @@
>
> static uint64_t vhost_req_user_to_vdpa[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
> [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
> };
> @@ -248,6 +247,12 @@ vhost_vdpa_set_vring_kick(struct virtio_user_dev *dev,
> struct vhost_vring_file *
> return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_KICK, file);
> }
>
> +static int
> +vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev, struct
> vhost_vring_addr *addr)
> +{
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
> +}
> +
> /* with below features, vhost vdpa does not need to do the checksum and TSO,
> * these info will be passed to virtio_user through virtio net header.
> */
> @@ -357,6 +362,7 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
> .get_vring_base = vhost_vdpa_get_vring_base,
> .set_vring_call = vhost_vdpa_set_vring_call,
> .set_vring_kick = vhost_vdpa_set_vring_kick,
> + .set_vring_addr = vhost_vdpa_set_vring_addr,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index e4975838ea..48ca7a2548 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -81,7 +81,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t
> queue_sel)
> state.num |= (1 << 15);
> dev->ops->set_vring_base(dev, &state);
>
> - dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
> + dev->ops->set_vring_addr(dev, &addr);
>
> /* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
> * lastly because vhost depends on this msg to judge if
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops
2021-01-06 12:06 ` Xia, Chenbo
@ 2021-01-15 10:19 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:19 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 1:06 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 30/40] net/virtio: add Virtio-user vring address ops
>>
>> This patch introduces a new callback for setting
>> vrings addresses.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 1 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 32 +++++++++++++------
>> drivers/net/virtio/virtio_user/vhost_user.c | 29 +++++++++++++----
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++-
>> .../net/virtio/virtio_user/virtio_user_dev.c | 2 +-
>> 5 files changed, 55 insertions(+), 17 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index b296ee215d..956eb58728 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -113,6 +113,7 @@ struct virtio_user_backend_ops {
>> int (*get_vring_base)(struct virtio_user_dev *dev, struct
>> vhost_vring_state *state);
>> int (*set_vring_call)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> int (*set_vring_kick)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> + int (*set_vring_addr)(struct virtio_user_dev *dev, struct
>> vhost_vring_addr *addr);
>> int (*send_request)(struct virtio_user_dev *dev,
>> enum vhost_user_request req,
>> void *arg);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 1805aee7f7..8cd86b72c6 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -294,9 +294,30 @@ vhost_kernel_set_vring_call(struct virtio_user_dev *dev,
>> struct vhost_vring_file
>> return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
>> }
>>
>> +static int
>> +vhost_kernel_set_vring_addr(struct virtio_user_dev *dev, struct
>> vhost_vring_addr *addr)
>> +{
>> + int ret, fd;
>> + uint32_t index = addr->index;
>
> Better use 'unsigned int index' here? It can hardly cause problem but I think
> it's better to use the type in struct vhost_vring_addr.
Fixed, thanks.
Maxime
> Thanks,
> Chenbo
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (29 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 30/40] net/virtio: add Virtio-user vring address ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-06 12:09 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 32/40] net/virtio: remove useless request ops Maxime Coquelin
` (9 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch introduces new callbacks to
get and set the device status.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 14 ++
drivers/net/virtio/virtio_user/vhost_user.c | 121 +++++++++++++-----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++-
.../net/virtio/virtio_user/virtio_user_dev.c | 42 ++----
5 files changed, 129 insertions(+), 66 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 956eb58728..c85ba9a47b 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -114,6 +114,8 @@ struct virtio_user_backend_ops {
int (*set_vring_call)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
int (*set_vring_kick)(struct virtio_user_dev *dev, struct vhost_vring_file *file);
int (*set_vring_addr)(struct virtio_user_dev *dev, struct vhost_vring_addr *addr);
+ int (*get_status)(struct virtio_user_dev *dev, uint8_t *status);
+ int (*set_status)(struct virtio_user_dev *dev, uint8_t status);
int (*send_request)(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 8cd86b72c6..0b74f73c97 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -316,6 +316,18 @@ vhost_kernel_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr
return 0;
}
+static int
+vhost_kernel_get_status(struct virtio_user_dev *dev __rte_unused, uint8_t *status __rte_unused)
+{
+ return -ENOTSUP;
+}
+
+static int
+vhost_kernel_set_status(struct virtio_user_dev *dev __rte_unused, uint8_t status __rte_unused)
+{
+ return -ENOTSUP;
+}
+
static uint64_t vhost_req_user_to_kernel[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
};
@@ -487,6 +499,8 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.set_vring_call = vhost_kernel_set_vring_call,
.set_vring_kick = vhost_kernel_set_vring_kick,
.set_vring_addr = vhost_kernel_set_vring_addr,
+ .get_status = vhost_kernel_get_status,
+ .set_status = vhost_kernel_set_status,
.send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 59cf366683..d426b44fe7 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -544,13 +544,100 @@ vhost_user_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *
return 0;
}
+static int
+vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_GET_STATUS,
+ .flags = VHOST_USER_VERSION,
+ };
+
+ /*
+ * If features have not been negotiated, we don't know if the backend
+ * supports protocol features
+ */
+ if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
+ return -ENOTSUP;
+
+ /* Status protocol feature requires protocol features support */
+ if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
+ return -ENOTSUP;
+
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
+ return -ENOTSUP;
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to send request");
+ goto err;
+ }
+
+ ret = vhost_user_read(dev->vhostfd, &msg);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to recv request");
+ goto err;
+ }
+
+ if (msg.request != VHOST_USER_GET_STATUS) {
+ PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
+ goto err;
+ }
+
+ if (msg.size != sizeof(msg.payload.u64)) {
+ PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
+ goto err;
+ }
+
+ *status = (uint8_t)msg.payload.u64;
+
+ return 0;
+err:
+ PMD_DRV_LOG(ERR, "Failed to get device status");
+ return -1;
+}
+
+static int
+vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
+{
+ int ret;
+ struct vhost_user_msg msg = {
+ .request = VHOST_USER_SET_STATUS,
+ .flags = VHOST_USER_VERSION,
+ .size = sizeof(msg.payload.u64),
+ .payload.u64 = status,
+ };
+
+ if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
+ msg.flags |= VHOST_USER_NEED_REPLY_MASK;
+
+ /*
+ * If features have not been negotiated, we don't know if the backend
+ * supports protocol features
+ */
+ if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
+ return -ENOTSUP;
+
+ /* Status protocol feature requires protocol features support */
+ if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
+ return -ENOTSUP;
+
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
+ return -ENOTSUP;
+
+ ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "Failed to send get status request");
+ return -1;
+ }
+
+ return vhost_user_check_reply_ack(dev, &msg);
+}
static struct vhost_user_msg m;
const char * const vhost_msg_strings[] = {
[VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
- [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
- [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
};
static int
@@ -561,7 +648,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
int need_reply = 0;
- int has_reply_ack = 0;
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int vhostfd = dev->vhostfd;
@@ -573,31 +659,11 @@ vhost_user_sock(struct virtio_user_dev *dev,
if (dev->is_server && vhostfd < 0)
return -1;
- if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
- has_reply_ack = 1;
-
msg.request = req;
msg.flags = VHOST_USER_VERSION;
msg.size = 0;
switch (req) {
- case VHOST_USER_GET_STATUS:
- if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
- (!(dev->protocol_features &
- (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
- return -ENOTSUP;
- need_reply = 1;
- break;
-
- case VHOST_USER_SET_STATUS:
- if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
- (!(dev->protocol_features &
- (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
- return -ENOTSUP;
-
- if (has_reply_ack)
- msg.flags |= VHOST_USER_NEED_REPLY_MASK;
- /* Fallthrough */
case VHOST_USER_SET_LOG_BASE:
msg.payload.u64 = *((__u64 *)arg);
msg.size = sizeof(m.payload.u64);
@@ -644,13 +710,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
}
switch (req) {
- case VHOST_USER_GET_STATUS:
- if (msg.size != sizeof(m.payload.u64)) {
- PMD_DRV_LOG(ERR, "Received bad msg size");
- return -1;
- }
- *((__u64 *)arg) = msg.payload.u64;
- break;
default:
/* Reply-ack handling */
if (msg.size != sizeof(m.payload.u64)) {
@@ -784,6 +843,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_vring_call = vhost_user_set_vring_call,
.set_vring_kick = vhost_user_set_vring_kick,
.set_vring_addr = vhost_user_set_vring_addr,
+ .get_status = vhost_user_get_status,
+ .set_status = vhost_user_set_status,
.send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index e09e7c9fb8..f4331884c3 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -36,8 +36,6 @@
static uint64_t vhost_req_user_to_vdpa[] = {
[VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
- [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
- [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
};
/* no alignment requirement */
@@ -253,6 +251,18 @@ vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *
return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
}
+static int
+vhost_vdpa_get_status(struct virtio_user_dev *dev, uint8_t *status)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_GET_STATUS, status);
+}
+
+static int
+vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
+{
+ return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_STATUS, &status);
+}
+
/* with below features, vhost vdpa does not need to do the checksum and TSO,
* these info will be passed to virtio_user through virtio net header.
*/
@@ -363,6 +373,8 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.set_vring_call = vhost_vdpa_set_vring_call,
.set_vring_kick = vhost_vdpa_set_vring_kick,
.set_vring_addr = vhost_vdpa_set_vring_addr,
+ .get_status = vhost_vdpa_get_status,
+ .set_status = vhost_vdpa_set_status,
.send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 48ca7a2548..e1f016aa8c 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -805,21 +805,12 @@ int
virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status)
{
int ret;
- uint64_t arg = status;
pthread_mutex_lock(&dev->mutex);
dev->status = status;
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER)
- ret = dev->ops->send_request(dev,
- VHOST_USER_SET_STATUS, &arg);
- else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA)
- ret = dev->ops->send_request(dev,
- VHOST_USER_SET_STATUS, &status);
- else
- ret = -ENOTSUP;
-
+ ret = dev->ops->set_status(dev, status);
if (ret && ret != -ENOTSUP) {
- PMD_INIT_LOG(ERR, "VHOST_USER_SET_STATUS failed (%d): %s", ret,
+ PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
strerror(errno));
}
@@ -830,29 +821,13 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status)
int
virtio_user_dev_update_status(struct virtio_user_dev *dev)
{
- uint64_t ret;
+ int ret;
uint8_t status;
- int err;
pthread_mutex_lock(&dev->mutex);
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
- err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS, &ret);
- if (!err && ret > UINT8_MAX) {
- PMD_INIT_LOG(ERR, "Invalid VHOST_USER_GET_STATUS "
- "response 0x%" PRIx64 "\n", ret);
- err = -1;
- goto error;
- }
- status = ret;
- } else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA) {
- err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS,
- &status);
- } else {
- err = -ENOTSUP;
- }
-
- if (!err) {
+ ret = dev->ops->get_status(dev, &status);
+ if (!ret) {
dev->status = status;
PMD_INIT_LOG(DEBUG, "Updated Device Status(0x%08x):\n"
"\t-RESET: %u\n"
@@ -870,12 +845,11 @@ virtio_user_dev_update_status(struct virtio_user_dev *dev)
!!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK),
!!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
!!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
- } else if (err != -ENOTSUP) {
- PMD_INIT_LOG(ERR, "VHOST_USER_GET_STATUS failed (%d): %s", err,
+ } else if (ret != -ENOTSUP) {
+ PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
strerror(errno));
}
-error:
pthread_mutex_unlock(&dev->mutex);
- return err;
+ return ret;
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops
2020-12-20 21:13 ` [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops Maxime Coquelin
@ 2021-01-06 12:09 ` Xia, Chenbo
2021-01-15 10:48 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-06 12:09 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 31/40] net/virtio: add Virtio-user status ops
>
> This patch introduces new callbacks to
> get and set the device status.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 2 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 14 ++
> drivers/net/virtio/virtio_user/vhost_user.c | 121 +++++++++++++-----
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++-
> .../net/virtio/virtio_user/virtio_user_dev.c | 42 ++----
> 5 files changed, 129 insertions(+), 66 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index 956eb58728..c85ba9a47b 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -114,6 +114,8 @@ struct virtio_user_backend_ops {
> int (*set_vring_call)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> int (*set_vring_kick)(struct virtio_user_dev *dev, struct
> vhost_vring_file *file);
> int (*set_vring_addr)(struct virtio_user_dev *dev, struct
> vhost_vring_addr *addr);
> + int (*get_status)(struct virtio_user_dev *dev, uint8_t *status);
> + int (*set_status)(struct virtio_user_dev *dev, uint8_t status);
> int (*send_request)(struct virtio_user_dev *dev,
> enum vhost_user_request req,
> void *arg);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 8cd86b72c6..0b74f73c97 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -316,6 +316,18 @@ vhost_kernel_set_vring_addr(struct virtio_user_dev *dev,
> struct vhost_vring_addr
> return 0;
> }
>
> +static int
> +vhost_kernel_get_status(struct virtio_user_dev *dev __rte_unused, uint8_t
> *status __rte_unused)
> +{
> + return -ENOTSUP;
> +}
> +
> +static int
> +vhost_kernel_set_status(struct virtio_user_dev *dev __rte_unused, uint8_t
> status __rte_unused)
> +{
> + return -ENOTSUP;
> +}
> +
> static uint64_t vhost_req_user_to_kernel[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> };
> @@ -487,6 +499,8 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
> .set_vring_call = vhost_kernel_set_vring_call,
> .set_vring_kick = vhost_kernel_set_vring_kick,
> .set_vring_addr = vhost_kernel_set_vring_addr,
> + .get_status = vhost_kernel_get_status,
> + .set_status = vhost_kernel_set_status,
> .send_request = vhost_kernel_send_request,
> .enable_qp = vhost_kernel_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
> b/drivers/net/virtio/virtio_user/vhost_user.c
> index 59cf366683..d426b44fe7 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -544,13 +544,100 @@ vhost_user_set_vring_addr(struct virtio_user_dev *dev,
> struct vhost_vring_addr *
> return 0;
> }
>
> +static int
> +vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
> +{
> + int ret;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_GET_STATUS,
> + .flags = VHOST_USER_VERSION,
> + };
> +
> + /*
> + * If features have not been negotiated, we don't know if the backend
> + * supports protocol features
> + */
> + if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
> + return -ENOTSUP;
> +
> + /* Status protocol feature requires protocol features support */
> + if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
> + return -ENOTSUP;
> +
> + if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> + return -ENOTSUP;
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to send request");
> + goto err;
> + }
> +
> + ret = vhost_user_read(dev->vhostfd, &msg);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to recv request");
> + goto err;
> + }
> +
> + if (msg.request != VHOST_USER_GET_STATUS) {
> + PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
> + goto err;
> + }
> +
> + if (msg.size != sizeof(msg.payload.u64)) {
> + PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
Better use %u here? If you will change this, please change other logs in
vhost_user_get_protocol_features and vhost_user_get_features too. Sorry
that I miss them in previous patches...
> + goto err;
> + }
> +
> + *status = (uint8_t)msg.payload.u64;
> +
> + return 0;
> +err:
> + PMD_DRV_LOG(ERR, "Failed to get device status");
> + return -1;
> +}
> +
> +static int
> +vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
> +{
> + int ret;
> + struct vhost_user_msg msg = {
> + .request = VHOST_USER_SET_STATUS,
> + .flags = VHOST_USER_VERSION,
> + .size = sizeof(msg.payload.u64),
> + .payload.u64 = status,
> + };
> +
> + if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> + msg.flags |= VHOST_USER_NEED_REPLY_MASK;
Move the above check after checking device supports VHOST_USER_F_PROTOCOL_FEATURES?
Logically, it's meaningless to check we supports certain protocol feature if we do
not support protocol feature at all :)
Thanks,
Chenbo
> +
> + /*
> + * If features have not been negotiated, we don't know if the backend
> + * supports protocol features
> + */
> + if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
> + return -ENOTSUP;
> +
> + /* Status protocol feature requires protocol features support */
> + if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
> + return -ENOTSUP;
> +
> + if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> + return -ENOTSUP;
> +
> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "Failed to send get status request");
> + return -1;
> + }
> +
> + return vhost_user_check_reply_ack(dev, &msg);
> +}
>
> static struct vhost_user_msg m;
>
> const char * const vhost_msg_strings[] = {
> [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
> - [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
> - [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
> };
>
> static int
> @@ -561,7 +648,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> struct vhost_user_msg msg;
> struct vhost_vring_file *file = 0;
> int need_reply = 0;
> - int has_reply_ack = 0;
> int fds[VHOST_MEMORY_MAX_NREGIONS];
> int fd_num = 0;
> int vhostfd = dev->vhostfd;
> @@ -573,31 +659,11 @@ vhost_user_sock(struct virtio_user_dev *dev,
> if (dev->is_server && vhostfd < 0)
> return -1;
>
> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> - has_reply_ack = 1;
> -
> msg.request = req;
> msg.flags = VHOST_USER_VERSION;
> msg.size = 0;
>
> switch (req) {
> - case VHOST_USER_GET_STATUS:
> - if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
> - (!(dev->protocol_features &
> - (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
> - return -ENOTSUP;
> - need_reply = 1;
> - break;
> -
> - case VHOST_USER_SET_STATUS:
> - if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
> - (!(dev->protocol_features &
> - (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
> - return -ENOTSUP;
> -
> - if (has_reply_ack)
> - msg.flags |= VHOST_USER_NEED_REPLY_MASK;
> - /* Fallthrough */
> case VHOST_USER_SET_LOG_BASE:
> msg.payload.u64 = *((__u64 *)arg);
> msg.size = sizeof(m.payload.u64);
> @@ -644,13 +710,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
> }
>
> switch (req) {
> - case VHOST_USER_GET_STATUS:
> - if (msg.size != sizeof(m.payload.u64)) {
> - PMD_DRV_LOG(ERR, "Received bad msg size");
> - return -1;
> - }
> - *((__u64 *)arg) = msg.payload.u64;
> - break;
> default:
> /* Reply-ack handling */
> if (msg.size != sizeof(m.payload.u64)) {
> @@ -784,6 +843,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
> .set_vring_call = vhost_user_set_vring_call,
> .set_vring_kick = vhost_user_set_vring_kick,
> .set_vring_addr = vhost_user_set_vring_addr,
> + .get_status = vhost_user_get_status,
> + .set_status = vhost_user_set_status,
> .send_request = vhost_user_sock,
> .enable_qp = vhost_user_enable_queue_pair
> };
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index e09e7c9fb8..f4331884c3 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -36,8 +36,6 @@
>
> static uint64_t vhost_req_user_to_vdpa[] = {
> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> - [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
> - [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
> };
>
> /* no alignment requirement */
> @@ -253,6 +251,18 @@ vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev,
> struct vhost_vring_addr *
> return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
> }
>
> +static int
> +vhost_vdpa_get_status(struct virtio_user_dev *dev, uint8_t *status)
> +{
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_GET_STATUS, status);
> +}
> +
> +static int
> +vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
> +{
> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_STATUS, &status);
> +}
> +
> /* with below features, vhost vdpa does not need to do the checksum and TSO,
> * these info will be passed to virtio_user through virtio net header.
> */
> @@ -363,6 +373,8 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
> .set_vring_call = vhost_vdpa_set_vring_call,
> .set_vring_kick = vhost_vdpa_set_vring_kick,
> .set_vring_addr = vhost_vdpa_set_vring_addr,
> + .get_status = vhost_vdpa_get_status,
> + .set_status = vhost_vdpa_set_status,
> .send_request = vhost_vdpa_send_request,
> .enable_qp = vhost_vdpa_enable_queue_pair,
> .dma_map = vhost_vdpa_dma_map,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 48ca7a2548..e1f016aa8c 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -805,21 +805,12 @@ int
> virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status)
> {
> int ret;
> - uint64_t arg = status;
>
> pthread_mutex_lock(&dev->mutex);
> dev->status = status;
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER)
> - ret = dev->ops->send_request(dev,
> - VHOST_USER_SET_STATUS, &arg);
> - else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA)
> - ret = dev->ops->send_request(dev,
> - VHOST_USER_SET_STATUS, &status);
> - else
> - ret = -ENOTSUP;
> -
> + ret = dev->ops->set_status(dev, status);
> if (ret && ret != -ENOTSUP) {
> - PMD_INIT_LOG(ERR, "VHOST_USER_SET_STATUS failed (%d): %s", ret,
> + PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
> strerror(errno));
> }
>
> @@ -830,29 +821,13 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev,
> uint8_t status)
> int
> virtio_user_dev_update_status(struct virtio_user_dev *dev)
> {
> - uint64_t ret;
> + int ret;
> uint8_t status;
> - int err;
>
> pthread_mutex_lock(&dev->mutex);
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
> - err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS, &ret);
> - if (!err && ret > UINT8_MAX) {
> - PMD_INIT_LOG(ERR, "Invalid VHOST_USER_GET_STATUS "
> - "response 0x%" PRIx64 "\n", ret);
> - err = -1;
> - goto error;
> - }
>
> - status = ret;
> - } else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA) {
> - err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS,
> - &status);
> - } else {
> - err = -ENOTSUP;
> - }
> -
> - if (!err) {
> + ret = dev->ops->get_status(dev, &status);
> + if (!ret) {
> dev->status = status;
> PMD_INIT_LOG(DEBUG, "Updated Device Status(0x%08x):\n"
> "\t-RESET: %u\n"
> @@ -870,12 +845,11 @@ virtio_user_dev_update_status(struct virtio_user_dev
> *dev)
> !!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK),
> !!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
> !!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
> - } else if (err != -ENOTSUP) {
> - PMD_INIT_LOG(ERR, "VHOST_USER_GET_STATUS failed (%d): %s", err,
> + } else if (ret != -ENOTSUP) {
> + PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
> strerror(errno));
> }
>
> -error:
> pthread_mutex_unlock(&dev->mutex);
> - return err;
> + return ret;
> }
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops
2021-01-06 12:09 ` Xia, Chenbo
@ 2021-01-15 10:48 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 10:48 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/6/21 1:09 PM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 31/40] net/virtio: add Virtio-user status ops
>>
>> This patch introduces new callbacks to
>> get and set the device status.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 2 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 14 ++
>> drivers/net/virtio/virtio_user/vhost_user.c | 121 +++++++++++++-----
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++-
>> .../net/virtio/virtio_user/virtio_user_dev.c | 42 ++----
>> 5 files changed, 129 insertions(+), 66 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index 956eb58728..c85ba9a47b 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -114,6 +114,8 @@ struct virtio_user_backend_ops {
>> int (*set_vring_call)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> int (*set_vring_kick)(struct virtio_user_dev *dev, struct
>> vhost_vring_file *file);
>> int (*set_vring_addr)(struct virtio_user_dev *dev, struct
>> vhost_vring_addr *addr);
>> + int (*get_status)(struct virtio_user_dev *dev, uint8_t *status);
>> + int (*set_status)(struct virtio_user_dev *dev, uint8_t status);
>> int (*send_request)(struct virtio_user_dev *dev,
>> enum vhost_user_request req,
>> void *arg);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 8cd86b72c6..0b74f73c97 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -316,6 +316,18 @@ vhost_kernel_set_vring_addr(struct virtio_user_dev *dev,
>> struct vhost_vring_addr
>> return 0;
>> }
>>
>> +static int
>> +vhost_kernel_get_status(struct virtio_user_dev *dev __rte_unused, uint8_t
>> *status __rte_unused)
>> +{
>> + return -ENOTSUP;
>> +}
>> +
>> +static int
>> +vhost_kernel_set_status(struct virtio_user_dev *dev __rte_unused, uint8_t
>> status __rte_unused)
>> +{
>> + return -ENOTSUP;
>> +}
>> +
>> static uint64_t vhost_req_user_to_kernel[] = {
>> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
>> };
>> @@ -487,6 +499,8 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
>> .set_vring_call = vhost_kernel_set_vring_call,
>> .set_vring_kick = vhost_kernel_set_vring_kick,
>> .set_vring_addr = vhost_kernel_set_vring_addr,
>> + .get_status = vhost_kernel_get_status,
>> + .set_status = vhost_kernel_set_status,
>> .send_request = vhost_kernel_send_request,
>> .enable_qp = vhost_kernel_enable_queue_pair
>> };
>> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
>> b/drivers/net/virtio/virtio_user/vhost_user.c
>> index 59cf366683..d426b44fe7 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_user.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
>> @@ -544,13 +544,100 @@ vhost_user_set_vring_addr(struct virtio_user_dev *dev,
>> struct vhost_vring_addr *
>> return 0;
>> }
>>
>> +static int
>> +vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
>> +{
>> + int ret;
>> + struct vhost_user_msg msg = {
>> + .request = VHOST_USER_GET_STATUS,
>> + .flags = VHOST_USER_VERSION,
>> + };
>> +
>> + /*
>> + * If features have not been negotiated, we don't know if the backend
>> + * supports protocol features
>> + */
>> + if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
>> + return -ENOTSUP;
>> +
>> + /* Status protocol feature requires protocol features support */
>> + if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
>> + return -ENOTSUP;
>> +
>> + if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> + return -ENOTSUP;
>> +
>> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to send request");
>> + goto err;
>> + }
>> +
>> + ret = vhost_user_read(dev->vhostfd, &msg);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to recv request");
>> + goto err;
>> + }
>> +
>> + if (msg.request != VHOST_USER_GET_STATUS) {
>> + PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
>> + goto err;
>> + }
>> +
>> + if (msg.size != sizeof(msg.payload.u64)) {
>> + PMD_DRV_LOG(ERR, "Unexpected payload size (%d)", msg.size);
>
> Better use %u here? If you will change this, please change other logs in
> vhost_user_get_protocol_features and vhost_user_get_features too. Sorry
> that I miss them in previous patches...
No problem, fixing in other patches too.
>> + goto err;
>> + }
>> +
>> + *status = (uint8_t)msg.payload.u64;
>> +
>> + return 0;
>> +err:
>> + PMD_DRV_LOG(ERR, "Failed to get device status");
>> + return -1;
>> +}
>> +
>> +static int
>> +vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
>> +{
>> + int ret;
>> + struct vhost_user_msg msg = {
>> + .request = VHOST_USER_SET_STATUS,
>> + .flags = VHOST_USER_VERSION,
>> + .size = sizeof(msg.payload.u64),
>> + .payload.u64 = status,
>> + };
>> +
>> + if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> + msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>
> Move the above check after checking device supports VHOST_USER_F_PROTOCOL_FEATURES?
> Logically, it's meaningless to check we supports certain protocol feature if we do
> not support protocol feature at all :)
Makes perfect sense, fixed.
Maxime
> Thanks,
> Chenbo
>
>> +
>> + /*
>> + * If features have not been negotiated, we don't know if the backend
>> + * supports protocol features
>> + */
>> + if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK))
>> + return -ENOTSUP;
>> +
>> + /* Status protocol feature requires protocol features support */
>> + if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
>> + return -ENOTSUP;
>> +
>> + if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> + return -ENOTSUP;
>> +
>> + ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to send get status request");
>> + return -1;
>> + }
>> +
>> + return vhost_user_check_reply_ack(dev, &msg);
>> +}
>>
>> static struct vhost_user_msg m;
>>
>> const char * const vhost_msg_strings[] = {
>> [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
>> - [VHOST_USER_SET_STATUS] = "VHOST_SET_STATUS",
>> - [VHOST_USER_GET_STATUS] = "VHOST_GET_STATUS",
>> };
>>
>> static int
>> @@ -561,7 +648,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
>> struct vhost_user_msg msg;
>> struct vhost_vring_file *file = 0;
>> int need_reply = 0;
>> - int has_reply_ack = 0;
>> int fds[VHOST_MEMORY_MAX_NREGIONS];
>> int fd_num = 0;
>> int vhostfd = dev->vhostfd;
>> @@ -573,31 +659,11 @@ vhost_user_sock(struct virtio_user_dev *dev,
>> if (dev->is_server && vhostfd < 0)
>> return -1;
>>
>> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> - has_reply_ack = 1;
>> -
>> msg.request = req;
>> msg.flags = VHOST_USER_VERSION;
>> msg.size = 0;
>>
>> switch (req) {
>> - case VHOST_USER_GET_STATUS:
>> - if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
>> - (!(dev->protocol_features &
>> - (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
>> - return -ENOTSUP;
>> - need_reply = 1;
>> - break;
>> -
>> - case VHOST_USER_SET_STATUS:
>> - if (!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK) ||
>> - (!(dev->protocol_features &
>> - (1ULL << VHOST_USER_PROTOCOL_F_STATUS))))
>> - return -ENOTSUP;
>> -
>> - if (has_reply_ack)
>> - msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>> - /* Fallthrough */
>> case VHOST_USER_SET_LOG_BASE:
>> msg.payload.u64 = *((__u64 *)arg);
>> msg.size = sizeof(m.payload.u64);
>> @@ -644,13 +710,6 @@ vhost_user_sock(struct virtio_user_dev *dev,
>> }
>>
>> switch (req) {
>> - case VHOST_USER_GET_STATUS:
>> - if (msg.size != sizeof(m.payload.u64)) {
>> - PMD_DRV_LOG(ERR, "Received bad msg size");
>> - return -1;
>> - }
>> - *((__u64 *)arg) = msg.payload.u64;
>> - break;
>> default:
>> /* Reply-ack handling */
>> if (msg.size != sizeof(m.payload.u64)) {
>> @@ -784,6 +843,8 @@ struct virtio_user_backend_ops virtio_ops_user = {
>> .set_vring_call = vhost_user_set_vring_call,
>> .set_vring_kick = vhost_user_set_vring_kick,
>> .set_vring_addr = vhost_user_set_vring_addr,
>> + .get_status = vhost_user_get_status,
>> + .set_status = vhost_user_set_status,
>> .send_request = vhost_user_sock,
>> .enable_qp = vhost_user_enable_queue_pair
>> };
>> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> b/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> index e09e7c9fb8..f4331884c3 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> @@ -36,8 +36,6 @@
>>
>> static uint64_t vhost_req_user_to_vdpa[] = {
>> [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
>> - [VHOST_USER_SET_STATUS] = VHOST_VDPA_SET_STATUS,
>> - [VHOST_USER_GET_STATUS] = VHOST_VDPA_GET_STATUS,
>> };
>>
>> /* no alignment requirement */
>> @@ -253,6 +251,18 @@ vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev,
>> struct vhost_vring_addr *
>> return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
>> }
>>
>> +static int
>> +vhost_vdpa_get_status(struct virtio_user_dev *dev, uint8_t *status)
>> +{
>> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_GET_STATUS, status);
>> +}
>> +
>> +static int
>> +vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
>> +{
>> + return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_STATUS, &status);
>> +}
>> +
>> /* with below features, vhost vdpa does not need to do the checksum and TSO,
>> * these info will be passed to virtio_user through virtio net header.
>> */
>> @@ -363,6 +373,8 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
>> .set_vring_call = vhost_vdpa_set_vring_call,
>> .set_vring_kick = vhost_vdpa_set_vring_kick,
>> .set_vring_addr = vhost_vdpa_set_vring_addr,
>> + .get_status = vhost_vdpa_get_status,
>> + .set_status = vhost_vdpa_set_status,
>> .send_request = vhost_vdpa_send_request,
>> .enable_qp = vhost_vdpa_enable_queue_pair,
>> .dma_map = vhost_vdpa_dma_map,
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index 48ca7a2548..e1f016aa8c 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -805,21 +805,12 @@ int
>> virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status)
>> {
>> int ret;
>> - uint64_t arg = status;
>>
>> pthread_mutex_lock(&dev->mutex);
>> dev->status = status;
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER)
>> - ret = dev->ops->send_request(dev,
>> - VHOST_USER_SET_STATUS, &arg);
>> - else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA)
>> - ret = dev->ops->send_request(dev,
>> - VHOST_USER_SET_STATUS, &status);
>> - else
>> - ret = -ENOTSUP;
>> -
>> + ret = dev->ops->set_status(dev, status);
>> if (ret && ret != -ENOTSUP) {
>> - PMD_INIT_LOG(ERR, "VHOST_USER_SET_STATUS failed (%d): %s", ret,
>> + PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
>> strerror(errno));
>> }
>>
>> @@ -830,29 +821,13 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev,
>> uint8_t status)
>> int
>> virtio_user_dev_update_status(struct virtio_user_dev *dev)
>> {
>> - uint64_t ret;
>> + int ret;
>> uint8_t status;
>> - int err;
>>
>> pthread_mutex_lock(&dev->mutex);
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
>> - err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS, &ret);
>> - if (!err && ret > UINT8_MAX) {
>> - PMD_INIT_LOG(ERR, "Invalid VHOST_USER_GET_STATUS "
>> - "response 0x%" PRIx64 "\n", ret);
>> - err = -1;
>> - goto error;
>> - }
>>
>> - status = ret;
>> - } else if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_VDPA) {
>> - err = dev->ops->send_request(dev, VHOST_USER_GET_STATUS,
>> - &status);
>> - } else {
>> - err = -ENOTSUP;
>> - }
>> -
>> - if (!err) {
>> + ret = dev->ops->get_status(dev, &status);
>> + if (!ret) {
>> dev->status = status;
>> PMD_INIT_LOG(DEBUG, "Updated Device Status(0x%08x):\n"
>> "\t-RESET: %u\n"
>> @@ -870,12 +845,11 @@ virtio_user_dev_update_status(struct virtio_user_dev
>> *dev)
>> !!(dev->status & VIRTIO_CONFIG_STATUS_FEATURES_OK),
>> !!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
>> !!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
>> - } else if (err != -ENOTSUP) {
>> - PMD_INIT_LOG(ERR, "VHOST_USER_GET_STATUS failed (%d): %s", err,
>> + } else if (ret != -ENOTSUP) {
>> + PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
>> strerror(errno));
>> }
>>
>> -error:
>> pthread_mutex_unlock(&dev->mutex);
>> - return err;
>> + return ret;
>> }
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 32/40] net/virtio: remove useless request ops
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (30 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 31/40] net/virtio: add Virtio-user status ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-20 21:13 ` [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling Maxime Coquelin
` (8 subsequent siblings)
40 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Now that all the ops have been implemented, we
can remove the send_request ops for all backends.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 15 +--
drivers/net/virtio/virtio_user/vhost_kernel.c | 43 ---------
drivers/net/virtio/virtio_user/vhost_user.c | 96 -------------------
drivers/net/virtio/virtio_user/vhost_vdpa.c | 49 ----------
4 files changed, 3 insertions(+), 200 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index c85ba9a47b..3e39fd2246 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -88,8 +88,6 @@ enum vhost_user_request {
VHOST_USER_MAX
};
-extern const char * const vhost_msg_strings[VHOST_USER_MAX];
-
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
@@ -116,16 +114,9 @@ struct virtio_user_backend_ops {
int (*set_vring_addr)(struct virtio_user_dev *dev, struct vhost_vring_addr *addr);
int (*get_status)(struct virtio_user_dev *dev, uint8_t *status);
int (*set_status)(struct virtio_user_dev *dev, uint8_t status);
- int (*send_request)(struct virtio_user_dev *dev,
- enum vhost_user_request req,
- void *arg);
- int (*enable_qp)(struct virtio_user_dev *dev,
- uint16_t pair_idx,
- int enable);
- int (*dma_map)(struct virtio_user_dev *dev, void *addr,
- uint64_t iova, size_t len);
- int (*dma_unmap)(struct virtio_user_dev *dev, void *addr,
- uint64_t iova, size_t len);
+ int (*enable_qp)(struct virtio_user_dev *dev, uint16_t pair_idx, int enable);
+ int (*dma_map)(struct virtio_user_dev *dev, void *addr, uint64_t iova, size_t len);
+ int (*dma_unmap)(struct virtio_user_dev *dev, void *addr, uint64_t iova, size_t len);
};
extern struct virtio_user_backend_ops virtio_ops_user;
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 0b74f73c97..c4c9c9d085 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -328,48 +328,6 @@ vhost_kernel_set_status(struct virtio_user_dev *dev __rte_unused, uint8_t status
return -ENOTSUP;
}
-static uint64_t vhost_req_user_to_kernel[] = {
- [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
-};
-
-static int
-vhost_kernel_send_request(struct virtio_user_dev *dev,
- enum vhost_user_request req,
- void *arg)
-{
- int ret = -1;
- unsigned int i;
- uint64_t req_kernel;
- int vhostfd;
-
- PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
-
- req_kernel = vhost_req_user_to_kernel[req];
-
- switch (req_kernel) {
- default:
- vhostfd = -1;
- }
- if (vhostfd == -1) {
- for (i = 0; i < dev->max_queue_pairs; ++i) {
- if (dev->vhostfds[i] < 0)
- continue;
-
- ret = ioctl(dev->vhostfds[i], req_kernel, arg);
- if (ret < 0)
- break;
- }
- } else {
- ret = ioctl(vhostfd, req_kernel, arg);
- }
-
- if (ret < 0)
- PMD_DRV_LOG(ERR, "%s failed: %s",
- vhost_msg_strings[req], strerror(errno));
-
- return ret;
-}
-
/**
* Set up environment to talk with a vhost kernel backend.
*
@@ -501,6 +459,5 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.set_vring_addr = vhost_kernel_set_vring_addr,
.get_status = vhost_kernel_get_status,
.set_status = vhost_kernel_set_status,
- .send_request = vhost_kernel_send_request,
.enable_qp = vhost_kernel_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index d426b44fe7..0632b80050 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -634,101 +634,6 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
return vhost_user_check_reply_ack(dev, &msg);
}
-static struct vhost_user_msg m;
-
-const char * const vhost_msg_strings[] = {
- [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
-};
-
-static int
-vhost_user_sock(struct virtio_user_dev *dev,
- enum vhost_user_request req,
- void *arg)
-{
- struct vhost_user_msg msg;
- struct vhost_vring_file *file = 0;
- int need_reply = 0;
- int fds[VHOST_MEMORY_MAX_NREGIONS];
- int fd_num = 0;
- int vhostfd = dev->vhostfd;
-
- RTE_SET_USED(m);
-
- PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
-
- if (dev->is_server && vhostfd < 0)
- return -1;
-
- msg.request = req;
- msg.flags = VHOST_USER_VERSION;
- msg.size = 0;
-
- switch (req) {
- case VHOST_USER_SET_LOG_BASE:
- msg.payload.u64 = *((__u64 *)arg);
- msg.size = sizeof(m.payload.u64);
- break;
-
- case VHOST_USER_RESET_OWNER:
- break;
-
- case VHOST_USER_SET_LOG_FD:
- fds[fd_num++] = *((int *)arg);
- break;
-
- case VHOST_USER_SET_VRING_ERR:
- file = arg;
- msg.payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
- msg.size = sizeof(m.payload.u64);
- if (file->fd > 0)
- fds[fd_num++] = file->fd;
- else
- msg.payload.u64 |= VHOST_USER_VRING_NOFD_MASK;
- break;
-
- default:
- PMD_DRV_LOG(ERR, "trying to send unhandled msg type");
- return -1;
- }
-
- if (vhost_user_write(vhostfd, &msg, fds, fd_num) < 0) {
- PMD_DRV_LOG(ERR, "%s failed: %s",
- vhost_msg_strings[req], strerror(errno));
- return -1;
- }
-
- if (need_reply || msg.flags & VHOST_USER_NEED_REPLY_MASK) {
- if (vhost_user_read(vhostfd, &msg) < 0) {
- PMD_DRV_LOG(ERR, "Received msg failed: %s",
- strerror(errno));
- return -1;
- }
-
- if (req != msg.request) {
- PMD_DRV_LOG(ERR, "Received unexpected msg type");
- return -1;
- }
-
- switch (req) {
- default:
- /* Reply-ack handling */
- if (msg.size != sizeof(m.payload.u64)) {
- PMD_DRV_LOG(ERR, "Received bad msg size");
- return -1;
- }
-
- if (msg.payload.u64 != 0) {
- PMD_DRV_LOG(ERR, "Slave replied NACK");
- return -1;
- }
-
- break;
- }
- }
-
- return 0;
-}
-
#define MAX_VIRTIO_USER_BACKLOG 1
static int
virtio_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
@@ -845,6 +750,5 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_vring_addr = vhost_user_set_vring_addr,
.get_status = vhost_user_get_status,
.set_status = vhost_user_set_status,
- .send_request = vhost_user_sock,
.enable_qp = vhost_user_enable_queue_pair
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index f4331884c3..ecfaeaf814 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -34,10 +34,6 @@
#define VHOST_VDPA_SET_STATUS _IOW(VHOST_VIRTIO, 0x72, __u8)
#define VHOST_VDPA_SET_VRING_ENABLE _IOW(VHOST_VIRTIO, 0x75, struct vhost_vring_state)
-static uint64_t vhost_req_user_to_vdpa[] = {
- [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
-};
-
/* no alignment requirement */
struct vhost_iotlb_msg {
uint64_t iova;
@@ -263,50 +259,6 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_STATUS, &status);
}
-/* with below features, vhost vdpa does not need to do the checksum and TSO,
- * these info will be passed to virtio_user through virtio net header.
- */
-#define VHOST_VDPA_GUEST_OFFLOADS_MASK \
- ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
- (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
- (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
- (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
- (1ULL << VIRTIO_NET_F_GUEST_UFO))
-
-#define VHOST_VDPA_HOST_OFFLOADS_MASK \
- ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
- (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
- (1ULL << VIRTIO_NET_F_CSUM))
-
-static int
-vhost_vdpa_send_request(struct virtio_user_dev *dev,
- enum vhost_user_request req,
- void *arg)
-{
- int ret = -1;
- uint64_t req_vdpa;
-
- PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
-
- req_vdpa = vhost_req_user_to_vdpa[req];
-
- switch (req_vdpa) {
- case VHOST_SET_VRING_ADDR:
- PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
- dev->vhostfd, *(unsigned int *)arg);
- break;
- default:
- break;
- }
-
- ret = ioctl(dev->vhostfd, req_vdpa, arg);
- if (ret < 0)
- PMD_DRV_LOG(ERR, "%s failed: %s",
- vhost_msg_strings[req], strerror(errno));
-
- return ret;
-}
-
/**
* Set up environment to talk with a vhost vdpa backend.
*
@@ -375,7 +327,6 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.set_vring_addr = vhost_vdpa_set_vring_addr,
.get_status = vhost_vdpa_get_status,
.set_status = vhost_vdpa_set_status,
- .send_request = vhost_vdpa_send_request,
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
.dma_unmap = vhost_vdpa_dma_unmap,
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (31 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 32/40] net/virtio: remove useless request ops Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2021-01-07 2:26 ` Xia, Chenbo
2020-12-20 21:13 ` [dpdk-dev] [PATCH 34/40] net/virtio: move Vhost-user reqs to Vhost-user backend Maxime Coquelin
` (7 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch adds error logs and propagate errors reported
by the backend. It also adds the device path in error logs
to make it easier to distinguish which device is facing the
issue.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
.../net/virtio/virtio_user/virtio_user_dev.c | 155 ++++++++++++------
1 file changed, 104 insertions(+), 51 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e1f016aa8c..b92b7f7aae 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -37,10 +37,15 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
* pair.
*/
struct vhost_vring_file file;
+ int ret;
file.index = queue_sel;
file.fd = dev->callfds[queue_sel];
- dev->ops->set_vring_call(dev, &file);
+ ret = dev->ops->set_vring_call(dev, &file);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to create queue %u\n", dev->path, queue_sel);
+ return -1;
+ }
return 0;
}
@@ -48,6 +53,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
static int
virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
{
+ int ret;
struct vhost_vring_file file;
struct vhost_vring_state state;
struct vring *vring = &dev->vrings[queue_sel];
@@ -73,15 +79,21 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- dev->ops->set_vring_num(dev, &state);
+ ret = dev->ops->set_vring_num(dev, &state);
+ if (ret < 0)
+ goto err;
state.index = queue_sel;
state.num = 0; /* no reservation */
if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
state.num |= (1 << 15);
- dev->ops->set_vring_base(dev, &state);
+ ret = dev->ops->set_vring_base(dev, &state);
+ if (ret < 0)
+ goto err;
- dev->ops->set_vring_addr(dev, &addr);
+ ret = dev->ops->set_vring_addr(dev, &addr);
+ if (ret < 0)
+ goto err;
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -89,9 +101,15 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
*/
file.index = queue_sel;
file.fd = dev->kickfds[queue_sel];
- dev->ops->set_vring_kick(dev, &file);
+ ret = dev->ops->set_vring_kick(dev, &file);
+ if (ret < 0)
+ goto err;
return 0;
+err:
+ PMD_INIT_LOG(ERR, "(%s) Failed to kick queue %u\n", dev->path, queue_sel);
+
+ return -1;
}
static int
@@ -103,14 +121,14 @@ virtio_user_queue_setup(struct virtio_user_dev *dev,
for (i = 0; i < dev->max_queue_pairs; ++i) {
queue_sel = 2 * i + VTNET_SQ_RQ_QUEUE_IDX;
if (fn(dev, queue_sel) < 0) {
- PMD_DRV_LOG(INFO, "setup rx vq fails: %u", i);
+ PMD_DRV_LOG(ERR, "(%s) setup rx vq fails: %u", dev->path, i);
return -1;
}
}
for (i = 0; i < dev->max_queue_pairs; ++i) {
queue_sel = 2 * i + VTNET_SQ_TQ_QUEUE_IDX;
if (fn(dev, queue_sel) < 0) {
- PMD_DRV_LOG(INFO, "setup tx vq fails: %u", i);
+ PMD_DRV_LOG(INFO, "(%s) setup tx vq fails: %u", dev->path, i);
return -1;
}
}
@@ -144,7 +162,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
ret = dev->ops->set_features(dev, features);
if (ret < 0)
goto error;
- PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
+ PMD_DRV_LOG(INFO, "(%s) set features: %" PRIx64, dev->path, features);
error:
pthread_mutex_unlock(&dev->mutex);
@@ -172,9 +190,10 @@ virtio_user_start_device(struct virtio_user_dev *dev)
rte_mcfg_mem_read_lock();
pthread_mutex_lock(&dev->mutex);
+ /* Vhost-user client not connected yet, will start later */
if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
dev->vhostfd < 0)
- goto error;
+ goto out;
/* Step 2: share memory regions */
ret = dev->ops->set_memory_table(dev);
@@ -182,15 +201,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
goto error;
/* Step 3: kick queues */
- if (virtio_user_queue_setup(dev, virtio_user_kick_queue) < 0)
+ ret = virtio_user_queue_setup(dev, virtio_user_kick_queue);
+ if (ret < 0)
goto error;
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- dev->ops->enable_qp(dev, 0, 1);
+ ret = dev->ops->enable_qp(dev, 0, 1);
+ if (ret < 0)
+ goto error;
dev->started = true;
+out:
pthread_mutex_unlock(&dev->mutex);
rte_mcfg_mem_read_unlock();
@@ -198,6 +221,9 @@ virtio_user_start_device(struct virtio_user_dev *dev)
error:
pthread_mutex_unlock(&dev->mutex);
rte_mcfg_mem_read_unlock();
+
+ PMD_INIT_LOG(ERR, "(%s) Failed to start device\n", dev->path);
+
/* TODO: free resource here or caller to check */
return -1;
}
@@ -206,31 +232,40 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
{
struct vhost_vring_state state;
uint32_t i;
- int error = 0;
+ int ret;
pthread_mutex_lock(&dev->mutex);
if (!dev->started)
goto out;
- for (i = 0; i < dev->max_queue_pairs; ++i)
- dev->ops->enable_qp(dev, i, 0);
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ ret = dev->ops->enable_qp(dev, i, 0);
+ if (ret < 0)
+ goto err;
+ }
/* Stop the backend. */
for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
state.index = i;
- if (dev->ops->get_vring_base(dev, &state) < 0) {
- PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
- i);
- error = -1;
- goto out;
+ ret = dev->ops->get_vring_base(dev, &state);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "(%s) get_vring_base failed, index=%u", dev->path, i);
+ goto err;
}
}
dev->started = false;
+
out:
pthread_mutex_unlock(&dev->mutex);
- return error;
+ return 0;
+err:
+ pthread_mutex_unlock(&dev->mutex);
+
+ PMD_INIT_LOG(ERR, "(%s) Failed to stop device\n", dev->path);
+
+ return -1;
}
static inline void
@@ -270,12 +305,12 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
*/
callfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
if (callfd < 0) {
- PMD_DRV_LOG(ERR, "callfd error, %s", strerror(errno));
+ PMD_DRV_LOG(ERR, "(%s) callfd error, %s", dev->path, strerror(errno));
break;
}
kickfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
if (kickfd < 0) {
- PMD_DRV_LOG(ERR, "kickfd error, %s", strerror(errno));
+ PMD_DRV_LOG(ERR, "(%s) kickfd error, %s", dev->path, strerror(errno));
break;
}
dev->callfds[i] = callfd;
@@ -303,7 +338,7 @@ virtio_user_fill_intr_handle(struct virtio_user_dev *dev)
if (!eth_dev->intr_handle) {
eth_dev->intr_handle = malloc(sizeof(*eth_dev->intr_handle));
if (!eth_dev->intr_handle) {
- PMD_DRV_LOG(ERR, "fail to allocate intr_handle");
+ PMD_DRV_LOG(ERR, "(%s) fail to allocate intr_handle", dev->path);
return -1;
}
memset(eth_dev->intr_handle, 0, sizeof(*eth_dev->intr_handle));
@@ -334,6 +369,7 @@ virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
struct virtio_user_dev *dev = arg;
struct rte_memseg_list *msl;
uint16_t i;
+ int ret = 0;
/* ignore externally allocated memory */
msl = rte_mem_virt2memseg_list(addr);
@@ -346,18 +382,29 @@ virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
goto exit;
/* Step 1: pause the active queues */
- for (i = 0; i < dev->queue_pairs; i++)
- dev->ops->enable_qp(dev, i, 0);
+ for (i = 0; i < dev->queue_pairs; i++) {
+ ret = dev->ops->enable_qp(dev, i, 0);
+ if (ret < 0)
+ goto exit;
+ }
/* Step 2: update memory regions */
- dev->ops->set_memory_table(dev);
+ ret = dev->ops->set_memory_table(dev);
+ if (ret < 0)
+ goto exit;
/* Step 3: resume the active queues */
- for (i = 0; i < dev->queue_pairs; i++)
- dev->ops->enable_qp(dev, i, 1);
+ for (i = 0; i < dev->queue_pairs; i++) {
+ ret = dev->ops->enable_qp(dev, i, 1);
+ if (ret < 0)
+ goto exit;
+ }
exit:
pthread_mutex_unlock(&dev->mutex);
+
+ if (ret < 0)
+ PMD_DRV_LOG(ERR, "(%s) Failed to update memory table\n", dev->path);
}
static int
@@ -387,7 +434,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
dev->tapfds = malloc(dev->max_queue_pairs *
sizeof(int));
if (!dev->vhostfds || !dev->tapfds) {
- PMD_INIT_LOG(ERR, "Failed to malloc");
+ PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
return -1;
}
@@ -399,19 +446,25 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
VIRTIO_USER_BACKEND_VHOST_VDPA) {
dev->ops = &virtio_ops_vdpa;
} else {
- PMD_DRV_LOG(ERR, "Unknown backend type");
+ PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
return -1;
}
}
- if (dev->ops->setup(dev) < 0)
+ if (dev->ops->setup(dev) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
return -1;
+ }
- if (virtio_user_dev_init_notify(dev) < 0)
+ if (virtio_user_dev_init_notify(dev) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to init notifiers\n", dev->path);
return -1;
+ }
- if (virtio_user_fill_intr_handle(dev) < 0)
+ if (virtio_user_fill_intr_handle(dev) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to init interrupt handler\n", dev->path);
return -1;
+ }
return 0;
}
@@ -472,7 +525,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
if (virtio_user_dev_setup(dev) < 0) {
- PMD_INIT_LOG(ERR, "backend set up fails");
+ PMD_INIT_LOG(ERR, "(%s) backend set up fails", dev->path);
return -1;
}
@@ -482,26 +535,29 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
if (!dev->is_server) {
if (dev->ops->set_owner(dev) < 0) {
- PMD_INIT_LOG(ERR, "set_owner fails: %s",
- strerror(errno));
+ PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev->path);
return -1;
}
if (dev->ops->get_features(dev, &dev->device_features) < 0) {
- PMD_INIT_LOG(ERR, "get_features failed: %s",
- strerror(errno));
+ PMD_INIT_LOG(ERR, "(%s) Failed to get backend features", dev->path);
return -1;
}
-
if (dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->get_protocol_features(dev, &protocol_features))
+ if (dev->ops->get_protocol_features(dev, &protocol_features)) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol features",
+ dev->path);
return -1;
+ }
dev->protocol_features &= protocol_features;
- if (dev->ops->set_protocol_features(dev, dev->protocol_features))
+ if (dev->ops->set_protocol_features(dev, dev->protocol_features)) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol features",
+ dev->path);
return -1;
+ }
if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
@@ -568,8 +624,8 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
if (rte_mem_event_callback_register(VIRTIO_USER_MEM_EVENT_CLB_NAME,
virtio_user_mem_event_cb, dev)) {
if (rte_errno != ENOTSUP) {
- PMD_INIT_LOG(ERR, "Failed to register mem event"
- " callback\n");
+ PMD_INIT_LOG(ERR, "(%s) Failed to register mem event callback\n",
+ dev->path);
return -1;
}
}
@@ -622,8 +678,8 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
uint8_t ret = 0;
if (q_pairs > dev->max_queue_pairs) {
- PMD_INIT_LOG(ERR, "multi-q config %u, but only %u supported",
- q_pairs, dev->max_queue_pairs);
+ PMD_INIT_LOG(ERR, "(%s) multi-q config %u, but only %u supported",
+ dev->path, q_pairs, dev->max_queue_pairs);
return -1;
}
@@ -809,10 +865,8 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status)
pthread_mutex_lock(&dev->mutex);
dev->status = status;
ret = dev->ops->set_status(dev, status);
- if (ret && ret != -ENOTSUP) {
- PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
- strerror(errno));
- }
+ if (ret && ret != -ENOTSUP)
+ PMD_INIT_LOG(ERR, "(%s) Failed to set backend status\n", dev->path);
pthread_mutex_unlock(&dev->mutex);
return ret;
@@ -846,8 +900,7 @@ virtio_user_dev_update_status(struct virtio_user_dev *dev)
!!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
!!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
} else if (ret != -ENOTSUP) {
- PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
- strerror(errno));
+ PMD_INIT_LOG(ERR, "(%s) Failed to get backend status\n", dev->path);
}
pthread_mutex_unlock(&dev->mutex);
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling
2020-12-20 21:13 ` [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling Maxime Coquelin
@ 2021-01-07 2:26 ` Xia, Chenbo
2021-01-15 11:09 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-07 2:26 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 33/40] net/virtio: improve Virtio-user errors handling
>
> This patch adds error logs and propagate errors reported
s/propagate/propagates
> by the backend. It also adds the device path in error logs
> to make it easier to distinguish which device is facing the
> issue.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> .../net/virtio/virtio_user/virtio_user_dev.c | 155 ++++++++++++------
> 1 file changed, 104 insertions(+), 51 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index e1f016aa8c..b92b7f7aae 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -37,10 +37,15 @@ virtio_user_create_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
> * pair.
> */
> struct vhost_vring_file file;
> + int ret;
>
> file.index = queue_sel;
> file.fd = dev->callfds[queue_sel];
> - dev->ops->set_vring_call(dev, &file);
> + ret = dev->ops->set_vring_call(dev, &file);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to create queue %u\n", dev->path,
> queue_sel);
> + return -1;
> + }
>
> return 0;
> }
> @@ -48,6 +53,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
> static int
> virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
> {
> + int ret;
> struct vhost_vring_file file;
> struct vhost_vring_state state;
> struct vring *vring = &dev->vrings[queue_sel];
> @@ -73,15 +79,21 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
>
> state.index = queue_sel;
> state.num = vring->num;
> - dev->ops->set_vring_num(dev, &state);
> + ret = dev->ops->set_vring_num(dev, &state);
> + if (ret < 0)
> + goto err;
>
> state.index = queue_sel;
> state.num = 0; /* no reservation */
> if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
> state.num |= (1 << 15);
> - dev->ops->set_vring_base(dev, &state);
> + ret = dev->ops->set_vring_base(dev, &state);
> + if (ret < 0)
> + goto err;
>
> - dev->ops->set_vring_addr(dev, &addr);
> + ret = dev->ops->set_vring_addr(dev, &addr);
> + if (ret < 0)
> + goto err;
>
> /* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
> * lastly because vhost depends on this msg to judge if
> @@ -89,9 +101,15 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
> uint32_t queue_sel)
> */
> file.index = queue_sel;
> file.fd = dev->kickfds[queue_sel];
> - dev->ops->set_vring_kick(dev, &file);
> + ret = dev->ops->set_vring_kick(dev, &file);
> + if (ret < 0)
> + goto err;
>
> return 0;
> +err:
> + PMD_INIT_LOG(ERR, "(%s) Failed to kick queue %u\n", dev->path,
> queue_sel);
> +
> + return -1;
> }
>
> static int
> @@ -103,14 +121,14 @@ virtio_user_queue_setup(struct virtio_user_dev *dev,
> for (i = 0; i < dev->max_queue_pairs; ++i) {
> queue_sel = 2 * i + VTNET_SQ_RQ_QUEUE_IDX;
> if (fn(dev, queue_sel) < 0) {
> - PMD_DRV_LOG(INFO, "setup rx vq fails: %u", i);
> + PMD_DRV_LOG(ERR, "(%s) setup rx vq fails: %u", dev->path, i);
Maybe 'setup rx vq %u fails' is better?
> return -1;
> }
> }
> for (i = 0; i < dev->max_queue_pairs; ++i) {
> queue_sel = 2 * i + VTNET_SQ_TQ_QUEUE_IDX;
> if (fn(dev, queue_sel) < 0) {
> - PMD_DRV_LOG(INFO, "setup tx vq fails: %u", i);
> + PMD_DRV_LOG(INFO, "(%s) setup tx vq fails: %u", dev->path,
> i);
Maybe 'setup tx vq %u fails' is better?
> return -1;
> }
> }
> @@ -144,7 +162,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
> ret = dev->ops->set_features(dev, features);
> if (ret < 0)
> goto error;
> - PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
> + PMD_DRV_LOG(INFO, "(%s) set features: %" PRIx64, dev->path, features);
Add '0x' before '%" PRIx64'?
> error:
> pthread_mutex_unlock(&dev->mutex);
>
> @@ -172,9 +190,10 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> rte_mcfg_mem_read_lock();
> pthread_mutex_lock(&dev->mutex);
>
> + /* Vhost-user client not connected yet, will start later */
> if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
> dev->vhostfd < 0)
> - goto error;
> + goto out;
>
> /* Step 2: share memory regions */
> ret = dev->ops->set_memory_table(dev);
> @@ -182,15 +201,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> goto error;
>
> /* Step 3: kick queues */
> - if (virtio_user_queue_setup(dev, virtio_user_kick_queue) < 0)
> + ret = virtio_user_queue_setup(dev, virtio_user_kick_queue);
> + if (ret < 0)
> goto error;
>
> /* Step 4: enable queues
> * we enable the 1st queue pair by default.
> */
> - dev->ops->enable_qp(dev, 0, 1);
> + ret = dev->ops->enable_qp(dev, 0, 1);
> + if (ret < 0)
> + goto error;
>
> dev->started = true;
> +out:
> pthread_mutex_unlock(&dev->mutex);
> rte_mcfg_mem_read_unlock();
>
> @@ -198,6 +221,9 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> error:
> pthread_mutex_unlock(&dev->mutex);
> rte_mcfg_mem_read_unlock();
> +
> + PMD_INIT_LOG(ERR, "(%s) Failed to start device\n", dev->path);
> +
> /* TODO: free resource here or caller to check */
> return -1;
> }
> @@ -206,31 +232,40 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
> {
> struct vhost_vring_state state;
> uint32_t i;
> - int error = 0;
> + int ret;
>
> pthread_mutex_lock(&dev->mutex);
> if (!dev->started)
> goto out;
>
> - for (i = 0; i < dev->max_queue_pairs; ++i)
> - dev->ops->enable_qp(dev, i, 0);
> + for (i = 0; i < dev->max_queue_pairs; ++i) {
> + ret = dev->ops->enable_qp(dev, i, 0);
> + if (ret < 0)
> + goto err;
> + }
>
> /* Stop the backend. */
> for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
> state.index = i;
> - if (dev->ops->get_vring_base(dev, &state) < 0) {
> - PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
> - i);
> - error = -1;
> - goto out;
> + ret = dev->ops->get_vring_base(dev, &state);
> + if (ret < 0) {
> + PMD_DRV_LOG(ERR, "(%s) get_vring_base failed, index=%u",
> dev->path, i);
> + goto err;
> }
> }
>
> dev->started = false;
> +
> out:
> pthread_mutex_unlock(&dev->mutex);
>
> - return error;
> + return 0;
> +err:
> + pthread_mutex_unlock(&dev->mutex);
> +
> + PMD_INIT_LOG(ERR, "(%s) Failed to stop device\n", dev->path);
> +
> + return -1;
> }
>
> static inline void
> @@ -270,12 +305,12 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
> */
> callfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
> if (callfd < 0) {
> - PMD_DRV_LOG(ERR, "callfd error, %s", strerror(errno));
> + PMD_DRV_LOG(ERR, "(%s) callfd error, %s", dev->path,
> strerror(errno));
> break;
> }
> kickfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
> if (kickfd < 0) {
> - PMD_DRV_LOG(ERR, "kickfd error, %s", strerror(errno));
> + PMD_DRV_LOG(ERR, "(%s) kickfd error, %s", dev->path,
> strerror(errno));
> break;
> }
> dev->callfds[i] = callfd;
> @@ -303,7 +338,7 @@ virtio_user_fill_intr_handle(struct virtio_user_dev *dev)
> if (!eth_dev->intr_handle) {
> eth_dev->intr_handle = malloc(sizeof(*eth_dev->intr_handle));
> if (!eth_dev->intr_handle) {
> - PMD_DRV_LOG(ERR, "fail to allocate intr_handle");
> + PMD_DRV_LOG(ERR, "(%s) fail to allocate intr_handle", dev-
> >path);
s/fail/failed ?
Thanks,
Chenbo
> return -1;
> }
> memset(eth_dev->intr_handle, 0, sizeof(*eth_dev->intr_handle));
> @@ -334,6 +369,7 @@ virtio_user_mem_event_cb(enum rte_mem_event type
> __rte_unused,
> struct virtio_user_dev *dev = arg;
> struct rte_memseg_list *msl;
> uint16_t i;
> + int ret = 0;
>
> /* ignore externally allocated memory */
> msl = rte_mem_virt2memseg_list(addr);
> @@ -346,18 +382,29 @@ virtio_user_mem_event_cb(enum rte_mem_event type
> __rte_unused,
> goto exit;
>
> /* Step 1: pause the active queues */
> - for (i = 0; i < dev->queue_pairs; i++)
> - dev->ops->enable_qp(dev, i, 0);
> + for (i = 0; i < dev->queue_pairs; i++) {
> + ret = dev->ops->enable_qp(dev, i, 0);
> + if (ret < 0)
> + goto exit;
> + }
>
> /* Step 2: update memory regions */
> - dev->ops->set_memory_table(dev);
> + ret = dev->ops->set_memory_table(dev);
> + if (ret < 0)
> + goto exit;
>
> /* Step 3: resume the active queues */
> - for (i = 0; i < dev->queue_pairs; i++)
> - dev->ops->enable_qp(dev, i, 1);
> + for (i = 0; i < dev->queue_pairs; i++) {
> + ret = dev->ops->enable_qp(dev, i, 1);
> + if (ret < 0)
> + goto exit;
> + }
>
> exit:
> pthread_mutex_unlock(&dev->mutex);
> +
> + if (ret < 0)
> + PMD_DRV_LOG(ERR, "(%s) Failed to update memory table\n", dev-
> >path);
> }
>
> static int
> @@ -387,7 +434,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> dev->tapfds = malloc(dev->max_queue_pairs *
> sizeof(int));
> if (!dev->vhostfds || !dev->tapfds) {
> - PMD_INIT_LOG(ERR, "Failed to malloc");
> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev-
> >path);
> return -1;
> }
>
> @@ -399,19 +446,25 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> VIRTIO_USER_BACKEND_VHOST_VDPA) {
> dev->ops = &virtio_ops_vdpa;
> } else {
> - PMD_DRV_LOG(ERR, "Unknown backend type");
> + PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
> return -1;
> }
> }
>
> - if (dev->ops->setup(dev) < 0)
> + if (dev->ops->setup(dev) < 0) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
> return -1;
> + }
>
> - if (virtio_user_dev_init_notify(dev) < 0)
> + if (virtio_user_dev_init_notify(dev) < 0) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to init notifiers\n", dev->path);
> return -1;
> + }
>
> - if (virtio_user_fill_intr_handle(dev) < 0)
> + if (virtio_user_fill_intr_handle(dev) < 0) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to init interrupt handler\n", dev-
> >path);
> return -1;
> + }
>
> return 0;
> }
> @@ -472,7 +525,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
> *path, int queues,
> }
>
> if (virtio_user_dev_setup(dev) < 0) {
> - PMD_INIT_LOG(ERR, "backend set up fails");
> + PMD_INIT_LOG(ERR, "(%s) backend set up fails", dev->path);
> return -1;
> }
>
> @@ -482,26 +535,29 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
> *path, int queues,
>
> if (!dev->is_server) {
> if (dev->ops->set_owner(dev) < 0) {
> - PMD_INIT_LOG(ERR, "set_owner fails: %s",
> - strerror(errno));
> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev-
> >path);
> return -1;
> }
>
> if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> - PMD_INIT_LOG(ERR, "get_features failed: %s",
> - strerror(errno));
> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend features",
> dev->path);
> return -1;
> }
>
> -
> if (dev->device_features & (1ULL <<
> VHOST_USER_F_PROTOCOL_FEATURES)) {
> - if (dev->ops->get_protocol_features(dev, &protocol_features))
> + if (dev->ops->get_protocol_features(dev, &protocol_features))
> {
> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol
> features",
> + dev->path);
> return -1;
> + }
>
> dev->protocol_features &= protocol_features;
>
> - if (dev->ops->set_protocol_features(dev, dev-
> >protocol_features))
> + if (dev->ops->set_protocol_features(dev, dev-
> >protocol_features)) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol
> features",
> + dev->path);
> return -1;
> + }
>
> if (!(dev->protocol_features & (1ULL <<
> VHOST_USER_PROTOCOL_F_MQ)))
> dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
> @@ -568,8 +624,8 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
> *path, int queues,
> if (rte_mem_event_callback_register(VIRTIO_USER_MEM_EVENT_CLB_NAME,
> virtio_user_mem_event_cb, dev)) {
> if (rte_errno != ENOTSUP) {
> - PMD_INIT_LOG(ERR, "Failed to register mem event"
> - " callback\n");
> + PMD_INIT_LOG(ERR, "(%s) Failed to register mem event
> callback\n",
> + dev->path);
> return -1;
> }
> }
> @@ -622,8 +678,8 @@ virtio_user_handle_mq(struct virtio_user_dev *dev,
> uint16_t q_pairs)
> uint8_t ret = 0;
>
> if (q_pairs > dev->max_queue_pairs) {
> - PMD_INIT_LOG(ERR, "multi-q config %u, but only %u supported",
> - q_pairs, dev->max_queue_pairs);
> + PMD_INIT_LOG(ERR, "(%s) multi-q config %u, but only %u supported",
> + dev->path, q_pairs, dev->max_queue_pairs);
> return -1;
> }
>
> @@ -809,10 +865,8 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev,
> uint8_t status)
> pthread_mutex_lock(&dev->mutex);
> dev->status = status;
> ret = dev->ops->set_status(dev, status);
> - if (ret && ret != -ENOTSUP) {
> - PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
> - strerror(errno));
> - }
> + if (ret && ret != -ENOTSUP)
> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend status\n", dev-
> >path);
>
> pthread_mutex_unlock(&dev->mutex);
> return ret;
> @@ -846,8 +900,7 @@ virtio_user_dev_update_status(struct virtio_user_dev *dev)
> !!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
> !!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
> } else if (ret != -ENOTSUP) {
> - PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
> - strerror(errno));
> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend status\n", dev-
> >path);
> }
>
> pthread_mutex_unlock(&dev->mutex);
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling
2021-01-07 2:26 ` Xia, Chenbo
@ 2021-01-15 11:09 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 11:09 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/7/21 3:26 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 33/40] net/virtio: improve Virtio-user errors handling
>>
>> This patch adds error logs and propagate errors reported
>
> s/propagate/propagates
Fixed.
>> by the backend. It also adds the device path in error logs
>> to make it easier to distinguish which device is facing the
>> issue.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> .../net/virtio/virtio_user/virtio_user_dev.c | 155 ++++++++++++------
>> 1 file changed, 104 insertions(+), 51 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index e1f016aa8c..b92b7f7aae 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -37,10 +37,15 @@ virtio_user_create_queue(struct virtio_user_dev *dev,
>> uint32_t queue_sel)
>> * pair.
>> */
>> struct vhost_vring_file file;
>> + int ret;
>>
>> file.index = queue_sel;
>> file.fd = dev->callfds[queue_sel];
>> - dev->ops->set_vring_call(dev, &file);
>> + ret = dev->ops->set_vring_call(dev, &file);
>> + if (ret < 0) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to create queue %u\n", dev->path,
>> queue_sel);
>> + return -1;
>> + }
>>
>> return 0;
>> }
>> @@ -48,6 +53,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev,
>> uint32_t queue_sel)
>> static int
>> virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
>> {
>> + int ret;
>> struct vhost_vring_file file;
>> struct vhost_vring_state state;
>> struct vring *vring = &dev->vrings[queue_sel];
>> @@ -73,15 +79,21 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
>> uint32_t queue_sel)
>>
>> state.index = queue_sel;
>> state.num = vring->num;
>> - dev->ops->set_vring_num(dev, &state);
>> + ret = dev->ops->set_vring_num(dev, &state);
>> + if (ret < 0)
>> + goto err;
>>
>> state.index = queue_sel;
>> state.num = 0; /* no reservation */
>> if (dev->features & (1ULL << VIRTIO_F_RING_PACKED))
>> state.num |= (1 << 15);
>> - dev->ops->set_vring_base(dev, &state);
>> + ret = dev->ops->set_vring_base(dev, &state);
>> + if (ret < 0)
>> + goto err;
>>
>> - dev->ops->set_vring_addr(dev, &addr);
>> + ret = dev->ops->set_vring_addr(dev, &addr);
>> + if (ret < 0)
>> + goto err;
>>
>> /* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
>> * lastly because vhost depends on this msg to judge if
>> @@ -89,9 +101,15 @@ virtio_user_kick_queue(struct virtio_user_dev *dev,
>> uint32_t queue_sel)
>> */
>> file.index = queue_sel;
>> file.fd = dev->kickfds[queue_sel];
>> - dev->ops->set_vring_kick(dev, &file);
>> + ret = dev->ops->set_vring_kick(dev, &file);
>> + if (ret < 0)
>> + goto err;
>>
>> return 0;
>> +err:
>> + PMD_INIT_LOG(ERR, "(%s) Failed to kick queue %u\n", dev->path,
>> queue_sel);
>> +
>> + return -1;
>> }
>>
>> static int
>> @@ -103,14 +121,14 @@ virtio_user_queue_setup(struct virtio_user_dev *dev,
>> for (i = 0; i < dev->max_queue_pairs; ++i) {
>> queue_sel = 2 * i + VTNET_SQ_RQ_QUEUE_IDX;
>> if (fn(dev, queue_sel) < 0) {
>> - PMD_DRV_LOG(INFO, "setup rx vq fails: %u", i);
>> + PMD_DRV_LOG(ERR, "(%s) setup rx vq fails: %u", dev->path, i);
>
> Maybe 'setup rx vq %u fails' is better?
>
>> return -1;
>> }
>> }
>> for (i = 0; i < dev->max_queue_pairs; ++i) {
>> queue_sel = 2 * i + VTNET_SQ_TQ_QUEUE_IDX;
>> if (fn(dev, queue_sel) < 0) {
>> - PMD_DRV_LOG(INFO, "setup tx vq fails: %u", i);
>> + PMD_DRV_LOG(INFO, "(%s) setup tx vq fails: %u", dev->path,
>> i);
>
> Maybe 'setup tx vq %u fails' is better?
Fixes both to "setup tx vq %u failed".
>> return -1;
>> }
>> }
>> @@ -144,7 +162,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>> ret = dev->ops->set_features(dev, features);
>> if (ret < 0)
>> goto error;
>> - PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
>> + PMD_DRV_LOG(INFO, "(%s) set features: %" PRIx64, dev->path, features);
>
> Add '0x' before '%" PRIx64'?
>
>> error:
>> pthread_mutex_unlock(&dev->mutex);
>>
>> @@ -172,9 +190,10 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>> rte_mcfg_mem_read_lock();
>> pthread_mutex_lock(&dev->mutex);
>>
>> + /* Vhost-user client not connected yet, will start later */
>> if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
>> dev->vhostfd < 0)
>> - goto error;
>> + goto out;
>>
>> /* Step 2: share memory regions */
>> ret = dev->ops->set_memory_table(dev);
>> @@ -182,15 +201,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>> goto error;
>>
>> /* Step 3: kick queues */
>> - if (virtio_user_queue_setup(dev, virtio_user_kick_queue) < 0)
>> + ret = virtio_user_queue_setup(dev, virtio_user_kick_queue);
>> + if (ret < 0)
>> goto error;
>>
>> /* Step 4: enable queues
>> * we enable the 1st queue pair by default.
>> */
>> - dev->ops->enable_qp(dev, 0, 1);
>> + ret = dev->ops->enable_qp(dev, 0, 1);
>> + if (ret < 0)
>> + goto error;
>>
>> dev->started = true;
>> +out:
>> pthread_mutex_unlock(&dev->mutex);
>> rte_mcfg_mem_read_unlock();
>>
>> @@ -198,6 +221,9 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>> error:
>> pthread_mutex_unlock(&dev->mutex);
>> rte_mcfg_mem_read_unlock();
>> +
>> + PMD_INIT_LOG(ERR, "(%s) Failed to start device\n", dev->path);
>> +
>> /* TODO: free resource here or caller to check */
>> return -1;
>> }
>> @@ -206,31 +232,40 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
>> {
>> struct vhost_vring_state state;
>> uint32_t i;
>> - int error = 0;
>> + int ret;
>>
>> pthread_mutex_lock(&dev->mutex);
>> if (!dev->started)
>> goto out;
>>
>> - for (i = 0; i < dev->max_queue_pairs; ++i)
>> - dev->ops->enable_qp(dev, i, 0);
>> + for (i = 0; i < dev->max_queue_pairs; ++i) {
>> + ret = dev->ops->enable_qp(dev, i, 0);
>> + if (ret < 0)
>> + goto err;
>> + }
>>
>> /* Stop the backend. */
>> for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
>> state.index = i;
>> - if (dev->ops->get_vring_base(dev, &state) < 0) {
>> - PMD_DRV_LOG(ERR, "get_vring_base failed, index=%u",
>> - i);
>> - error = -1;
>> - goto out;
>> + ret = dev->ops->get_vring_base(dev, &state);
>> + if (ret < 0) {
>> + PMD_DRV_LOG(ERR, "(%s) get_vring_base failed, index=%u",
>> dev->path, i);
>> + goto err;
>> }
>> }
>>
>> dev->started = false;
>> +
>> out:
>> pthread_mutex_unlock(&dev->mutex);
>>
>> - return error;
>> + return 0;
>> +err:
>> + pthread_mutex_unlock(&dev->mutex);
>> +
>> + PMD_INIT_LOG(ERR, "(%s) Failed to stop device\n", dev->path);
>> +
>> + return -1;
>> }
>>
>> static inline void
>> @@ -270,12 +305,12 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
>> */
>> callfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
>> if (callfd < 0) {
>> - PMD_DRV_LOG(ERR, "callfd error, %s", strerror(errno));
>> + PMD_DRV_LOG(ERR, "(%s) callfd error, %s", dev->path,
>> strerror(errno));
>> break;
>> }
>> kickfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
>> if (kickfd < 0) {
>> - PMD_DRV_LOG(ERR, "kickfd error, %s", strerror(errno));
>> + PMD_DRV_LOG(ERR, "(%s) kickfd error, %s", dev->path,
>> strerror(errno));
>> break;
>> }
>> dev->callfds[i] = callfd;
>> @@ -303,7 +338,7 @@ virtio_user_fill_intr_handle(struct virtio_user_dev *dev)
>> if (!eth_dev->intr_handle) {
>> eth_dev->intr_handle = malloc(sizeof(*eth_dev->intr_handle));
>> if (!eth_dev->intr_handle) {
>> - PMD_DRV_LOG(ERR, "fail to allocate intr_handle");
>> + PMD_DRV_LOG(ERR, "(%s) fail to allocate intr_handle", dev-
>>> path);
>
> s/fail/failed ?
Indeed.
Thanks,
Maxime
> Thanks,
> Chenbo
>
>> return -1;
>> }
>> memset(eth_dev->intr_handle, 0, sizeof(*eth_dev->intr_handle));
>> @@ -334,6 +369,7 @@ virtio_user_mem_event_cb(enum rte_mem_event type
>> __rte_unused,
>> struct virtio_user_dev *dev = arg;
>> struct rte_memseg_list *msl;
>> uint16_t i;
>> + int ret = 0;
>>
>> /* ignore externally allocated memory */
>> msl = rte_mem_virt2memseg_list(addr);
>> @@ -346,18 +382,29 @@ virtio_user_mem_event_cb(enum rte_mem_event type
>> __rte_unused,
>> goto exit;
>>
>> /* Step 1: pause the active queues */
>> - for (i = 0; i < dev->queue_pairs; i++)
>> - dev->ops->enable_qp(dev, i, 0);
>> + for (i = 0; i < dev->queue_pairs; i++) {
>> + ret = dev->ops->enable_qp(dev, i, 0);
>> + if (ret < 0)
>> + goto exit;
>> + }
>>
>> /* Step 2: update memory regions */
>> - dev->ops->set_memory_table(dev);
>> + ret = dev->ops->set_memory_table(dev);
>> + if (ret < 0)
>> + goto exit;
>>
>> /* Step 3: resume the active queues */
>> - for (i = 0; i < dev->queue_pairs; i++)
>> - dev->ops->enable_qp(dev, i, 1);
>> + for (i = 0; i < dev->queue_pairs; i++) {
>> + ret = dev->ops->enable_qp(dev, i, 1);
>> + if (ret < 0)
>> + goto exit;
>> + }
>>
>> exit:
>> pthread_mutex_unlock(&dev->mutex);
>> +
>> + if (ret < 0)
>> + PMD_DRV_LOG(ERR, "(%s) Failed to update memory table\n", dev-
>>> path);
>> }
>>
>> static int
>> @@ -387,7 +434,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
>> dev->tapfds = malloc(dev->max_queue_pairs *
>> sizeof(int));
>> if (!dev->vhostfds || !dev->tapfds) {
>> - PMD_INIT_LOG(ERR, "Failed to malloc");
>> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev-
>>> path);
>> return -1;
>> }
>>
>> @@ -399,19 +446,25 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
>> VIRTIO_USER_BACKEND_VHOST_VDPA) {
>> dev->ops = &virtio_ops_vdpa;
>> } else {
>> - PMD_DRV_LOG(ERR, "Unknown backend type");
>> + PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
>> return -1;
>> }
>> }
>>
>> - if (dev->ops->setup(dev) < 0)
>> + if (dev->ops->setup(dev) < 0) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
>> return -1;
>> + }
>>
>> - if (virtio_user_dev_init_notify(dev) < 0)
>> + if (virtio_user_dev_init_notify(dev) < 0) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to init notifiers\n", dev->path);
>> return -1;
>> + }
>>
>> - if (virtio_user_fill_intr_handle(dev) < 0)
>> + if (virtio_user_fill_intr_handle(dev) < 0) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to init interrupt handler\n", dev-
>>> path);
>> return -1;
>> + }
>>
>> return 0;
>> }
>> @@ -472,7 +525,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>> *path, int queues,
>> }
>>
>> if (virtio_user_dev_setup(dev) < 0) {
>> - PMD_INIT_LOG(ERR, "backend set up fails");
>> + PMD_INIT_LOG(ERR, "(%s) backend set up fails", dev->path);
>> return -1;
>> }
>>
>> @@ -482,26 +535,29 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>> *path, int queues,
>>
>> if (!dev->is_server) {
>> if (dev->ops->set_owner(dev) < 0) {
>> - PMD_INIT_LOG(ERR, "set_owner fails: %s",
>> - strerror(errno));
>> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev-
>>> path);
>> return -1;
>> }
>>
>> if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>> - PMD_INIT_LOG(ERR, "get_features failed: %s",
>> - strerror(errno));
>> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend features",
>> dev->path);
>> return -1;
>> }
>>
>> -
>> if (dev->device_features & (1ULL <<
>> VHOST_USER_F_PROTOCOL_FEATURES)) {
>> - if (dev->ops->get_protocol_features(dev, &protocol_features))
>> + if (dev->ops->get_protocol_features(dev, &protocol_features))
>> {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol
>> features",
>> + dev->path);
>> return -1;
>> + }
>>
>> dev->protocol_features &= protocol_features;
>>
>> - if (dev->ops->set_protocol_features(dev, dev-
>>> protocol_features))
>> + if (dev->ops->set_protocol_features(dev, dev-
>>> protocol_features)) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol
>> features",
>> + dev->path);
>> return -1;
>> + }
>>
>> if (!(dev->protocol_features & (1ULL <<
>> VHOST_USER_PROTOCOL_F_MQ)))
>> dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
>> @@ -568,8 +624,8 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char
>> *path, int queues,
>> if (rte_mem_event_callback_register(VIRTIO_USER_MEM_EVENT_CLB_NAME,
>> virtio_user_mem_event_cb, dev)) {
>> if (rte_errno != ENOTSUP) {
>> - PMD_INIT_LOG(ERR, "Failed to register mem event"
>> - " callback\n");
>> + PMD_INIT_LOG(ERR, "(%s) Failed to register mem event
>> callback\n",
>> + dev->path);
>> return -1;
>> }
>> }
>> @@ -622,8 +678,8 @@ virtio_user_handle_mq(struct virtio_user_dev *dev,
>> uint16_t q_pairs)
>> uint8_t ret = 0;
>>
>> if (q_pairs > dev->max_queue_pairs) {
>> - PMD_INIT_LOG(ERR, "multi-q config %u, but only %u supported",
>> - q_pairs, dev->max_queue_pairs);
>> + PMD_INIT_LOG(ERR, "(%s) multi-q config %u, but only %u supported",
>> + dev->path, q_pairs, dev->max_queue_pairs);
>> return -1;
>> }
>>
>> @@ -809,10 +865,8 @@ virtio_user_dev_set_status(struct virtio_user_dev *dev,
>> uint8_t status)
>> pthread_mutex_lock(&dev->mutex);
>> dev->status = status;
>> ret = dev->ops->set_status(dev, status);
>> - if (ret && ret != -ENOTSUP) {
>> - PMD_INIT_LOG(ERR, "Virtio-user set status failed (%d): %s", ret,
>> - strerror(errno));
>> - }
>> + if (ret && ret != -ENOTSUP)
>> + PMD_INIT_LOG(ERR, "(%s) Failed to set backend status\n", dev-
>>> path);
>>
>> pthread_mutex_unlock(&dev->mutex);
>> return ret;
>> @@ -846,8 +900,7 @@ virtio_user_dev_update_status(struct virtio_user_dev *dev)
>> !!(dev->status & VIRTIO_CONFIG_STATUS_DEV_NEED_RESET),
>> !!(dev->status & VIRTIO_CONFIG_STATUS_FAILED));
>> } else if (ret != -ENOTSUP) {
>> - PMD_INIT_LOG(ERR, "Virtio-user get status failed (%d): %s", ret,
>> - strerror(errno));
>> + PMD_INIT_LOG(ERR, "(%s) Failed to get backend status\n", dev-
>>> path);
>> }
>>
>> pthread_mutex_unlock(&dev->mutex);
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 34/40] net/virtio: move Vhost-user reqs to Vhost-user backend
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (32 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 33/40] net/virtio: improve Virtio-user errors handling Maxime Coquelin
@ 2020-12-20 21:13 ` Maxime Coquelin
2020-12-20 21:14 ` [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking Maxime Coquelin
` (6 subsequent siblings)
40 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:13 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Now that we have a proper isolation of the backends,
we can move Vhost-user requests declaration to the
Vhost-user backend file.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 25 ---------------------
drivers/net/virtio/virtio_user/vhost_user.c | 25 +++++++++++++++++++++
2 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 3e39fd2246..4fba203cc5 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -63,31 +63,6 @@ struct vhost_vring_addr {
#define VHOST_USER_PROTOCOL_F_STATUS 16
#endif
-enum vhost_user_request {
- VHOST_USER_NONE = 0,
- VHOST_USER_GET_FEATURES = 1,
- VHOST_USER_SET_FEATURES = 2,
- VHOST_USER_SET_OWNER = 3,
- VHOST_USER_RESET_OWNER = 4,
- VHOST_USER_SET_MEM_TABLE = 5,
- VHOST_USER_SET_LOG_BASE = 6,
- VHOST_USER_SET_LOG_FD = 7,
- VHOST_USER_SET_VRING_NUM = 8,
- VHOST_USER_SET_VRING_ADDR = 9,
- VHOST_USER_SET_VRING_BASE = 10,
- VHOST_USER_GET_VRING_BASE = 11,
- VHOST_USER_SET_VRING_KICK = 12,
- VHOST_USER_SET_VRING_CALL = 13,
- VHOST_USER_SET_VRING_ERR = 14,
- VHOST_USER_GET_PROTOCOL_FEATURES = 15,
- VHOST_USER_SET_PROTOCOL_FEATURES = 16,
- VHOST_USER_GET_QUEUE_NUM = 17,
- VHOST_USER_SET_VRING_ENABLE = 18,
- VHOST_USER_SET_STATUS = 39,
- VHOST_USER_GET_STATUS = 40,
- VHOST_USER_MAX
-};
-
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 0632b80050..a57106a468 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -27,6 +27,31 @@ struct vhost_memory {
struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
};
+enum vhost_user_request {
+ VHOST_USER_NONE = 0,
+ VHOST_USER_GET_FEATURES = 1,
+ VHOST_USER_SET_FEATURES = 2,
+ VHOST_USER_SET_OWNER = 3,
+ VHOST_USER_RESET_OWNER = 4,
+ VHOST_USER_SET_MEM_TABLE = 5,
+ VHOST_USER_SET_LOG_BASE = 6,
+ VHOST_USER_SET_LOG_FD = 7,
+ VHOST_USER_SET_VRING_NUM = 8,
+ VHOST_USER_SET_VRING_ADDR = 9,
+ VHOST_USER_SET_VRING_BASE = 10,
+ VHOST_USER_GET_VRING_BASE = 11,
+ VHOST_USER_SET_VRING_KICK = 12,
+ VHOST_USER_SET_VRING_CALL = 13,
+ VHOST_USER_SET_VRING_ERR = 14,
+ VHOST_USER_GET_PROTOCOL_FEATURES = 15,
+ VHOST_USER_SET_PROTOCOL_FEATURES = 16,
+ VHOST_USER_GET_QUEUE_NUM = 17,
+ VHOST_USER_SET_VRING_ENABLE = 18,
+ VHOST_USER_SET_STATUS = 39,
+ VHOST_USER_GET_STATUS = 40,
+ VHOST_USER_MAX
+};
+
struct vhost_user_msg {
enum vhost_user_request request;
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (33 preceding siblings ...)
2020-12-20 21:13 ` [dpdk-dev] [PATCH 34/40] net/virtio: move Vhost-user reqs to Vhost-user backend Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2021-01-07 3:20 ` Xia, Chenbo
2020-12-20 21:14 ` [dpdk-dev] [PATCH 36/40] net/virtio: move protocol features to Vhost-user Maxime Coquelin
` (5 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch makes the Vhost-user backend server mode
blocking at init, waiting for the client connection.
The goal is to make the driver more reliable, as without
waiting for client connection, the Virtio driver has to
assume the Vhost-user backend will support all the
features it has advertized, which could lead to undefined
behaviour.
For example, without this patch, if the user enables packed
ring Virtio feature but the backend does not support it,
the ring initialized by the driver will not be compatible
with the backend.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost_user.c | 9 +-
.../net/virtio/virtio_user/virtio_user_dev.c | 118 +++++++-----------
drivers/net/virtio/virtio_user_ethdev.c | 5 -
3 files changed, 54 insertions(+), 78 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index a57106a468..94a33326ae 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -677,6 +677,14 @@ virtio_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
if (ret < 0)
return -1;
+ PMD_DRV_LOG(NOTICE, "(%s) waiting for client connection...", dev->path);
+ dev->vhostfd = accept(fd, NULL, NULL);
+ if (dev->vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "Failed to accept initial client connection (%s)",
+ strerror(errno));
+ return -1;
+ }
+
flag = fcntl(fd, F_GETFL);
if (fcntl(fd, F_SETFL, flag | O_NONBLOCK) < 0) {
PMD_DRV_LOG(ERR, "fcntl failed, %s", strerror(errno));
@@ -721,7 +729,6 @@ vhost_user_setup(struct virtio_user_dev *dev)
close(fd);
return -1;
}
- dev->vhostfd = -1;
} else {
if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index b92b7f7aae..19d59d401e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -144,10 +144,6 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
pthread_mutex_lock(&dev->mutex);
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
- dev->vhostfd < 0)
- goto error;
-
/* Step 0: tell vhost to create queues */
if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
goto error;
@@ -190,11 +186,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
rte_mcfg_mem_read_lock();
pthread_mutex_lock(&dev->mutex);
- /* Vhost-user client not connected yet, will start later */
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
- dev->vhostfd < 0)
- goto out;
-
/* Step 2: share memory regions */
ret = dev->ops->set_memory_table(dev);
if (ret < 0)
@@ -213,7 +204,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
goto error;
dev->started = true;
-out:
+
pthread_mutex_unlock(&dev->mutex);
rte_mcfg_mem_read_unlock();
@@ -421,36 +412,36 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
return -1;
}
+ }
+
+ if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
dev->ops = &virtio_ops_user;
- } else {
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
- dev->ops = &virtio_ops_user;
- } else if (dev->backend_type ==
- VIRTIO_USER_BACKEND_VHOST_KERNEL) {
- dev->ops = &virtio_ops_kernel;
-
- dev->vhostfds = malloc(dev->max_queue_pairs *
- sizeof(int));
- dev->tapfds = malloc(dev->max_queue_pairs *
- sizeof(int));
- if (!dev->vhostfds || !dev->tapfds) {
- PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
- return -1;
- }
-
- for (q = 0; q < dev->max_queue_pairs; ++q) {
- dev->vhostfds[q] = -1;
- dev->tapfds[q] = -1;
- }
- } else if (dev->backend_type ==
- VIRTIO_USER_BACKEND_VHOST_VDPA) {
- dev->ops = &virtio_ops_vdpa;
- } else {
- PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
+ } else if (dev->backend_type ==
+ VIRTIO_USER_BACKEND_VHOST_KERNEL) {
+ dev->ops = &virtio_ops_kernel;
+
+ dev->vhostfds = malloc(dev->max_queue_pairs *
+ sizeof(int));
+ dev->tapfds = malloc(dev->max_queue_pairs *
+ sizeof(int));
+ if (!dev->vhostfds || !dev->tapfds) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
return -1;
}
+
+ for (q = 0; q < dev->max_queue_pairs; ++q) {
+ dev->vhostfds[q] = -1;
+ dev->tapfds[q] = -1;
+ }
+ } else if (dev->backend_type ==
+ VIRTIO_USER_BACKEND_VHOST_VDPA) {
+ dev->ops = &virtio_ops_vdpa;
+ } else {
+ PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
+ return -1;
}
+
if (dev->ops->setup(dev) < 0) {
PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
return -1;
@@ -533,52 +524,35 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->unsupported_features |=
(1ULL << VHOST_USER_F_PROTOCOL_FEATURES);
- if (!dev->is_server) {
- if (dev->ops->set_owner(dev) < 0) {
- PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev->path);
- return -1;
- }
+ if (dev->ops->set_owner(dev) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev->path);
+ return -1;
+ }
+
+ if (dev->ops->get_features(dev, &dev->device_features) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to get backend features", dev->path);
+ return -1;
+ }
- if (dev->ops->get_features(dev, &dev->device_features) < 0) {
- PMD_INIT_LOG(ERR, "(%s) Failed to get backend features", dev->path);
+ if (dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
+ if (dev->ops->get_protocol_features(dev, &protocol_features)) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol features",
+ dev->path);
return -1;
}
- if (dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->get_protocol_features(dev, &protocol_features)) {
- PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol features",
- dev->path);
- return -1;
- }
-
- dev->protocol_features &= protocol_features;
-
- if (dev->ops->set_protocol_features(dev, dev->protocol_features)) {
- PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol features",
- dev->path);
- return -1;
- }
+ dev->protocol_features &= protocol_features;
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
- dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
+ if (dev->ops->set_protocol_features(dev, dev->protocol_features)) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol features",
+ dev->path);
+ return -1;
}
- } else {
- /* We just pretend vhost-user can support all these features.
- * Note that this could be problematic that if some feature is
- * negotiated but not supported by the vhost-user which comes
- * later.
- */
- dev->device_features = VIRTIO_USER_SUPPORTED_FEATURES;
- /* We cannot assume VHOST_USER_PROTOCOL_F_STATUS is supported
- * until it's negotiated
- */
- dev->protocol_features &=
- ~(1ULL << VHOST_USER_PROTOCOL_F_STATUS);
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
+ dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
}
-
-
if (!mrg_rxbuf)
dev->unsupported_features |= (1ull << VIRTIO_NET_F_MRG_RXBUF);
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index c63d010e16..c4cfd3daed 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -174,11 +174,6 @@ virtio_user_delayed_handler(void *param)
if (dev->vhostfd >= 0) {
close(dev->vhostfd);
dev->vhostfd = -1;
- /* Until the featuers are negotiated again, don't assume
- * the backend supports VHOST_USER_PROTOCOL_F_STATUS
- */
- dev->protocol_features &=
- ~(1ULL << VHOST_USER_PROTOCOL_F_STATUS);
}
eth_dev->intr_handle->fd = dev->listenfd;
rte_intr_callback_register(eth_dev->intr_handle,
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking
2020-12-20 21:14 ` [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking Maxime Coquelin
@ 2021-01-07 3:20 ` Xia, Chenbo
2021-01-15 11:13 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-07 3:20 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 35/40] net/virtio: make server mode blocking
>
> This patch makes the Vhost-user backend server mode
> blocking at init, waiting for the client connection.
>
> The goal is to make the driver more reliable, as without
> waiting for client connection, the Virtio driver has to
> assume the Vhost-user backend will support all the
> features it has advertized, which could lead to undefined
> behaviour.
>
> For example, without this patch, if the user enables packed
> ring Virtio feature but the backend does not support it,
> the ring initialized by the driver will not be compatible
> with the backend.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_user.c | 9 +-
> .../net/virtio/virtio_user/virtio_user_dev.c | 118 +++++++-----------
> drivers/net/virtio/virtio_user_ethdev.c | 5 -
> 3 files changed, 54 insertions(+), 78 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
> b/drivers/net/virtio/virtio_user/vhost_user.c
> index a57106a468..94a33326ae 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -677,6 +677,14 @@ virtio_user_start_server(struct virtio_user_dev *dev,
> struct sockaddr_un *un)
> if (ret < 0)
> return -1;
>
> + PMD_DRV_LOG(NOTICE, "(%s) waiting for client connection...", dev->path);
> + dev->vhostfd = accept(fd, NULL, NULL);
> + if (dev->vhostfd < 0) {
> + PMD_DRV_LOG(ERR, "Failed to accept initial client connection (%s)",
> + strerror(errno));
> + return -1;
> + }
> +
> flag = fcntl(fd, F_GETFL);
> if (fcntl(fd, F_SETFL, flag | O_NONBLOCK) < 0) {
> PMD_DRV_LOG(ERR, "fcntl failed, %s", strerror(errno));
> @@ -721,7 +729,6 @@ vhost_user_setup(struct virtio_user_dev *dev)
> close(fd);
> return -1;
> }
> - dev->vhostfd = -1;
> } else {
> if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
> PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index b92b7f7aae..19d59d401e 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -144,10 +144,6 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>
> pthread_mutex_lock(&dev->mutex);
>
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
> - dev->vhostfd < 0)
> - goto error;
> -
> /* Step 0: tell vhost to create queues */
> if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
> goto error;
> @@ -190,11 +186,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> rte_mcfg_mem_read_lock();
> pthread_mutex_lock(&dev->mutex);
>
> - /* Vhost-user client not connected yet, will start later */
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
> - dev->vhostfd < 0)
> - goto out;
> -
> /* Step 2: share memory regions */
> ret = dev->ops->set_memory_table(dev);
> if (ret < 0)
> @@ -213,7 +204,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> goto error;
>
> dev->started = true;
> -out:
> +
> pthread_mutex_unlock(&dev->mutex);
> rte_mcfg_mem_read_unlock();
>
> @@ -421,36 +412,36 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
> return -1;
> }
> + }
> +
> + if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
> dev->ops = &virtio_ops_user;
> - } else {
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
> - dev->ops = &virtio_ops_user;
> - } else if (dev->backend_type ==
> - VIRTIO_USER_BACKEND_VHOST_KERNEL) {
> - dev->ops = &virtio_ops_kernel;
> -
> - dev->vhostfds = malloc(dev->max_queue_pairs *
> - sizeof(int));
> - dev->tapfds = malloc(dev->max_queue_pairs *
> - sizeof(int));
> - if (!dev->vhostfds || !dev->tapfds) {
> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev-
> >path);
> - return -1;
If tapfds failed on malloc, this will lead to vhostfds not freed?
And, should we free the fds when errors happen later in this function?
Thanks!
Chenbo
> - }
> -
> - for (q = 0; q < dev->max_queue_pairs; ++q) {
> - dev->vhostfds[q] = -1;
> - dev->tapfds[q] = -1;
> - }
> - } else if (dev->backend_type ==
> - VIRTIO_USER_BACKEND_VHOST_VDPA) {
> - dev->ops = &virtio_ops_vdpa;
> - } else {
> - PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking
2021-01-07 3:20 ` Xia, Chenbo
@ 2021-01-15 11:13 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 11:13 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/7/21 4:20 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 35/40] net/virtio: make server mode blocking
>>
>> This patch makes the Vhost-user backend server mode
>> blocking at init, waiting for the client connection.
>>
>> The goal is to make the driver more reliable, as without
>> waiting for client connection, the Virtio driver has to
>> assume the Vhost-user backend will support all the
>> features it has advertized, which could lead to undefined
>> behaviour.
>>
>> For example, without this patch, if the user enables packed
>> ring Virtio feature but the backend does not support it,
>> the ring initialized by the driver will not be compatible
>> with the backend.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost_user.c | 9 +-
>> .../net/virtio/virtio_user/virtio_user_dev.c | 118 +++++++-----------
>> drivers/net/virtio/virtio_user_ethdev.c | 5 -
>> 3 files changed, 54 insertions(+), 78 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
>> b/drivers/net/virtio/virtio_user/vhost_user.c
>> index a57106a468..94a33326ae 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_user.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
>> @@ -677,6 +677,14 @@ virtio_user_start_server(struct virtio_user_dev *dev,
>> struct sockaddr_un *un)
>> if (ret < 0)
>> return -1;
>>
>> + PMD_DRV_LOG(NOTICE, "(%s) waiting for client connection...", dev->path);
>> + dev->vhostfd = accept(fd, NULL, NULL);
>> + if (dev->vhostfd < 0) {
>> + PMD_DRV_LOG(ERR, "Failed to accept initial client connection (%s)",
>> + strerror(errno));
>> + return -1;
>> + }
>> +
>> flag = fcntl(fd, F_GETFL);
>> if (fcntl(fd, F_SETFL, flag | O_NONBLOCK) < 0) {
>> PMD_DRV_LOG(ERR, "fcntl failed, %s", strerror(errno));
>> @@ -721,7 +729,6 @@ vhost_user_setup(struct virtio_user_dev *dev)
>> close(fd);
>> return -1;
>> }
>> - dev->vhostfd = -1;
>> } else {
>> if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
>> PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index b92b7f7aae..19d59d401e 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -144,10 +144,6 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
>>
>> pthread_mutex_lock(&dev->mutex);
>>
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
>> - dev->vhostfd < 0)
>> - goto error;
>> -
>> /* Step 0: tell vhost to create queues */
>> if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
>> goto error;
>> @@ -190,11 +186,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>> rte_mcfg_mem_read_lock();
>> pthread_mutex_lock(&dev->mutex);
>>
>> - /* Vhost-user client not connected yet, will start later */
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER &&
>> - dev->vhostfd < 0)
>> - goto out;
>> -
>> /* Step 2: share memory regions */
>> ret = dev->ops->set_memory_table(dev);
>> if (ret < 0)
>> @@ -213,7 +204,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>> goto error;
>>
>> dev->started = true;
>> -out:
>> +
>> pthread_mutex_unlock(&dev->mutex);
>> rte_mcfg_mem_read_unlock();
>>
>> @@ -421,36 +412,36 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
>> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
>> return -1;
>> }
>> + }
>> +
>> + if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
>> dev->ops = &virtio_ops_user;
>> - } else {
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
>> - dev->ops = &virtio_ops_user;
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_KERNEL) {
>> - dev->ops = &virtio_ops_kernel;
>> -
>> - dev->vhostfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - dev->tapfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - if (!dev->vhostfds || !dev->tapfds) {
>> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev-
>>> path);
>> - return -1;
>
> If tapfds failed on malloc, this will lead to vhostfds not freed?
>
> And, should we free the fds when errors happen later in this function?
Certainly, but that it is not the purpose of this patch.
It is being moved to vhost-kernel in later patch, I'll check there if it
is freed properly.
Thanks,
Maxime
> Thanks!
> Chenbo
>
>> - }
>> -
>> - for (q = 0; q < dev->max_queue_pairs; ++q) {
>> - dev->vhostfds[q] = -1;
>> - dev->tapfds[q] = -1;
>> - }
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_VDPA) {
>> - dev->ops = &virtio_ops_vdpa;
>> - } else {
>> - PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 36/40] net/virtio: move protocol features to Vhost-user
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (34 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 35/40] net/virtio: make server mode blocking Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2020-12-20 21:14 ` [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data Maxime Coquelin
` (4 subsequent siblings)
40 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
Since only protocol features are specific to Vhost-user
backend, this patch moves all related code to Vhost-user
file.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 20 +----
drivers/net/virtio/virtio_user/vhost_kernel.c | 9 +++
drivers/net/virtio/virtio_user/vhost_user.c | 79 +++++++++++++++----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 9 +++
.../net/virtio/virtio_user/virtio_user_dev.c | 39 ++-------
.../net/virtio/virtio_user/virtio_user_dev.h | 5 +-
drivers/net/virtio/virtio_user_ethdev.c | 17 ----
7 files changed, 92 insertions(+), 86 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 4fba203cc5..b488ac78a1 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -46,23 +46,6 @@ struct vhost_vring_addr {
uint64_t log_guest_addr;
};
-#ifndef VHOST_USER_F_PROTOCOL_FEATURES
-#define VHOST_USER_F_PROTOCOL_FEATURES 30
-#endif
-
-/** Protocol features. */
-#ifndef VHOST_USER_PROTOCOL_F_MQ
-#define VHOST_USER_PROTOCOL_F_MQ 0
-#endif
-
-#ifndef VHOST_USER_PROTOCOL_F_REPLY_ACK
-#define VHOST_USER_PROTOCOL_F_REPLY_ACK 3
-#endif
-
-#ifndef VHOST_USER_PROTOCOL_F_STATUS
-#define VHOST_USER_PROTOCOL_F_STATUS 16
-#endif
-
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
@@ -74,11 +57,10 @@ struct virtio_user_dev;
struct virtio_user_backend_ops {
int (*setup)(struct virtio_user_dev *dev);
+ int (*get_backend_features)(uint64_t *features);
int (*set_owner)(struct virtio_user_dev *dev);
int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
int (*set_features)(struct virtio_user_dev *dev, uint64_t features);
- int (*get_protocol_features)(struct virtio_user_dev *dev, uint64_t *features);
- int (*set_protocol_features)(struct virtio_user_dev *dev, uint64_t features);
int (*set_memory_table)(struct virtio_user_dev *dev);
int (*set_vring_enable)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
int (*set_vring_num)(struct virtio_user_dev *dev, struct vhost_vring_state *state);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index c4c9c9d085..2fd00afa84 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -445,8 +445,17 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
return 0;
}
+static int
+vhost_kernel_get_backend_features(uint64_t *features)
+{
+ *features = 0;
+
+ return 0;
+}
+
struct virtio_user_backend_ops virtio_ops_kernel = {
.setup = vhost_kernel_setup,
+ .get_backend_features = vhost_kernel_get_backend_features,
.set_owner = vhost_kernel_set_owner,
.get_features = vhost_kernel_get_features,
.set_features = vhost_kernel_set_features,
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 94a33326ae..f67c40e047 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -17,6 +17,29 @@
#include "vhost.h"
#include "virtio_user_dev.h"
+
+#ifndef VHOST_USER_F_PROTOCOL_FEATURES
+#define VHOST_USER_F_PROTOCOL_FEATURES 30
+#endif
+
+/** Protocol features. */
+#ifndef VHOST_USER_PROTOCOL_F_MQ
+#define VHOST_USER_PROTOCOL_F_MQ 0
+#endif
+
+#ifndef VHOST_USER_PROTOCOL_F_REPLY_ACK
+#define VHOST_USER_PROTOCOL_F_REPLY_ACK 3
+#endif
+
+#ifndef VHOST_USER_PROTOCOL_F_STATUS
+#define VHOST_USER_PROTOCOL_F_STATUS 16
+#endif
+
+#define VHOST_USER_SUPPORTED_PROTOCOL_FEATURES \
+ (1ULL << VHOST_USER_PROTOCOL_F_MQ | \
+ 1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK | \
+ 1ULL << VHOST_USER_PROTOCOL_F_STATUS)
+
/* The version of the protocol we support */
#define VHOST_USER_VERSION 0x1
@@ -192,11 +215,11 @@ vhost_user_set_owner(struct virtio_user_dev *dev)
}
static int
-vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
+vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features)
{
int ret;
struct vhost_user_msg msg = {
- .request = VHOST_USER_GET_FEATURES,
+ .request = VHOST_USER_GET_PROTOCOL_FEATURES,
.flags = VHOST_USER_VERSION,
};
@@ -208,7 +231,7 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
if (ret < 0)
goto err;
- if (msg.request != VHOST_USER_GET_FEATURES) {
+ if (msg.request != VHOST_USER_GET_PROTOCOL_FEATURES) {
PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
goto err;
}
@@ -222,17 +245,17 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
return 0;
err:
- PMD_DRV_LOG(ERR, "Failed to get backend features");
+ PMD_DRV_LOG(ERR, "Failed to get backend protocol features");
return -1;
}
static int
-vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
+vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
{
int ret;
struct vhost_user_msg msg = {
- .request = VHOST_USER_SET_FEATURES,
+ .request = VHOST_USER_SET_PROTOCOL_FEATURES,
.flags = VHOST_USER_VERSION,
.size = sizeof(features),
.payload.u64 = features,
@@ -240,7 +263,7 @@ vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
if (ret < 0) {
- PMD_DRV_LOG(ERR, "Failed to set features");
+ PMD_DRV_LOG(ERR, "Failed to set protocol features");
return -1;
}
@@ -248,11 +271,11 @@ vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
}
static int
-vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features)
+vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
{
int ret;
struct vhost_user_msg msg = {
- .request = VHOST_USER_GET_PROTOCOL_FEATURES,
+ .request = VHOST_USER_GET_FEATURES,
.flags = VHOST_USER_VERSION,
};
@@ -264,7 +287,7 @@ vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features
if (ret < 0)
goto err;
- if (msg.request != VHOST_USER_GET_PROTOCOL_FEATURES) {
+ if (msg.request != VHOST_USER_GET_FEATURES) {
PMD_DRV_LOG(ERR, "Unexpected request type (%d)", msg.request);
goto err;
}
@@ -276,19 +299,36 @@ vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features
*features = msg.payload.u64;
+ if (!(*features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
+ return 0;
+
+ /* Negotiate protocol features */
+ ret = vhost_user_get_protocol_features(dev, &dev->protocol_features);
+ if (ret < 0)
+ goto err;
+
+ dev->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
+
+ ret = vhost_user_set_protocol_features(dev, dev->protocol_features);
+ if (ret < 0)
+ goto err;
+
+ if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
+ dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
+
return 0;
err:
- PMD_DRV_LOG(ERR, "Failed to get backend protocol features");
+ PMD_DRV_LOG(ERR, "Failed to get backend features");
return -1;
}
static int
-vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
+vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
{
int ret;
struct vhost_user_msg msg = {
- .request = VHOST_USER_SET_PROTOCOL_FEATURES,
+ .request = VHOST_USER_SET_FEATURES,
.flags = VHOST_USER_VERSION,
.size = sizeof(features),
.payload.u64 = features,
@@ -296,7 +336,7 @@ vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
if (ret < 0) {
- PMD_DRV_LOG(ERR, "Failed to set protocol features");
+ PMD_DRV_LOG(ERR, "Failed to set features");
return -1;
}
@@ -765,13 +805,20 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
return 0;
}
+static int
+vhost_user_get_backend_features(uint64_t *features)
+{
+ *features = 1ULL << VHOST_USER_F_PROTOCOL_FEATURES;
+
+ return 0;
+}
+
struct virtio_user_backend_ops virtio_ops_user = {
.setup = vhost_user_setup,
+ .get_backend_features = vhost_user_get_backend_features,
.set_owner = vhost_user_set_owner,
.get_features = vhost_user_get_features,
.set_features = vhost_user_set_features,
- .get_protocol_features = vhost_user_get_protocol_features,
- .set_protocol_features = vhost_user_set_protocol_features,
.set_memory_table = vhost_user_set_memory_table,
.set_vring_enable = vhost_user_set_vring_enable,
.set_vring_num = vhost_user_set_vring_num,
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index ecfaeaf814..c826f333e0 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -312,8 +312,17 @@ vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
return 0;
}
+static int
+vhost_vdpa_get_backend_features(uint64_t *features)
+{
+ *features = 0;
+
+ return 0;
+}
+
struct virtio_user_backend_ops virtio_ops_vdpa = {
.setup = vhost_vdpa_setup,
+ .get_backend_features = vhost_vdpa_get_backend_features,
.set_owner = vhost_vdpa_set_owner,
.get_features = vhost_vdpa_get_features,
.set_features = vhost_vdpa_set_features,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 19d59d401e..974de133bc 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -479,13 +479,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
1ULL << VIRTIO_NET_F_GUEST_TSO6 | \
1ULL << VIRTIO_F_IN_ORDER | \
1ULL << VIRTIO_F_VERSION_1 | \
- 1ULL << VIRTIO_F_RING_PACKED | \
- 1ULL << VHOST_USER_F_PROTOCOL_FEATURES)
-
-#define VIRTIO_USER_SUPPORTED_PROTOCOL_FEATURES \
- (1ULL << VHOST_USER_PROTOCOL_F_MQ | \
- 1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK | \
- 1ULL << VHOST_USER_PROTOCOL_F_STATUS)
+ 1ULL << VIRTIO_F_RING_PACKED)
int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
@@ -493,7 +487,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int server, int mrg_rxbuf, int in_order, int packed_vq,
enum virtio_user_backend_type backend_type)
{
- uint64_t protocol_features = 0;
+ uint64_t backend_features;
pthread_mutex_init(&dev->mutex, NULL);
strlcpy(dev->path, path, PATH_MAX);
@@ -504,8 +498,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->is_server = server;
dev->mac_specified = 0;
dev->frontend_features = 0;
- dev->unsupported_features = ~VIRTIO_USER_SUPPORTED_FEATURES;
- dev->protocol_features = VIRTIO_USER_SUPPORTED_PROTOCOL_FEATURES;
+ dev->unsupported_features = 0;
dev->backend_type = backend_type;
parse_mac(dev, mac);
@@ -520,37 +513,21 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
return -1;
}
- if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER)
- dev->unsupported_features |=
- (1ULL << VHOST_USER_F_PROTOCOL_FEATURES);
-
if (dev->ops->set_owner(dev) < 0) {
PMD_INIT_LOG(ERR, "(%s) Failed to set backend owner", dev->path);
return -1;
}
- if (dev->ops->get_features(dev, &dev->device_features) < 0) {
+ if (dev->ops->get_backend_features(&backend_features) < 0) {
PMD_INIT_LOG(ERR, "(%s) Failed to get backend features", dev->path);
return -1;
}
- if (dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->get_protocol_features(dev, &protocol_features)) {
- PMD_INIT_LOG(ERR, "(%s) Failed to get backend protocol features",
- dev->path);
- return -1;
- }
-
- dev->protocol_features &= protocol_features;
+ dev->unsupported_features = ~(VIRTIO_USER_SUPPORTED_FEATURES | backend_features);
- if (dev->ops->set_protocol_features(dev, dev->protocol_features)) {
- PMD_INIT_LOG(ERR, "(%s) Failed to set backend protocol features",
- dev->path);
- return -1;
- }
-
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
- dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
+ if (dev->ops->get_features(dev, &dev->device_features) < 0) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to get device features", dev->path);
+ return -1;
}
if (!mrg_rxbuf)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 0eb481ae21..b2da2944e9 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -30,6 +30,7 @@ struct virtio_user_dev {
int vhostfd;
int listenfd; /* listening fd */
bool is_server; /* server or client mode */
+ uint64_t protocol_features; /* negotiated protocol features*/
/* for vhost_kernel backend */
char *ifname;
@@ -49,9 +50,7 @@ struct virtio_user_dev {
uint64_t device_features; /* supported features by device */
uint64_t frontend_features; /* enabled frontend features */
uint64_t unsupported_features; /* unsupported features mask */
- uint64_t protocol_features; /* negotiated protocol features
- * (Vhost-user only)
- */
+
uint8_t status;
uint16_t net_status;
uint16_t port_id;
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index c4cfd3daed..77ad468768 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -70,7 +70,6 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
int ret, connectfd, old_status;
struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
struct virtio_hw *hw = &dev->hw;
- uint64_t protocol_features;
connectfd = accept(dev->listenfd, NULL, NULL);
if (connectfd < 0)
@@ -91,22 +90,6 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
return -1;
}
- if (dev->device_features &
- (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)) {
- if (dev->ops->get_protocol_features(dev, &protocol_features))
- return -1;
-
- /* Offer VHOST_USER_PROTOCOL_F_STATUS */
- dev->protocol_features |= (1ULL << VHOST_USER_PROTOCOL_F_STATUS);
- dev->protocol_features &= protocol_features;
-
- if (dev->ops->set_protocol_features(dev, dev->protocol_features))
- return -1;
-
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
- dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
- }
-
dev->device_features |= dev->frontend_features;
/* umask vhost-user unsupported features */
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (35 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 36/40] net/virtio: move protocol features to Vhost-user Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2021-01-05 21:26 ` David Marchand
2021-01-13 17:18 ` Adrian Moreno
2020-12-20 21:14 ` [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend Maxime Coquelin
` (3 subsequent siblings)
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
The goal of this patch is to introduce backend-specific
data in order to better isolate what is backend-specific
from what is generic to Virtio-user.
For now, only Vhost-user protocol features are moved to
Vhost-user backend data.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 1 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 7 +++
drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++----
drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++
.../net/virtio/virtio_user/virtio_user_dev.c | 2 +
.../net/virtio/virtio_user/virtio_user_dev.h | 3 +-
6 files changed, 58 insertions(+), 9 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index b488ac78a1..ee5598226d 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -57,6 +57,7 @@ struct virtio_user_dev;
struct virtio_user_backend_ops {
int (*setup)(struct virtio_user_dev *dev);
+ int (*destroy)(struct virtio_user_dev *dev);
int (*get_backend_features)(uint64_t *features);
int (*set_owner)(struct virtio_user_dev *dev);
int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 2fd00afa84..023fddcd69 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -357,6 +357,12 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
return 0;
}
+static int
+vhost_kernel_destroy(struct virtio_user_dev *dev)
+{
+ return 0;
+}
+
static int
vhost_kernel_set_backend(int vhostfd, int tapfd)
{
@@ -455,6 +461,7 @@ vhost_kernel_get_backend_features(uint64_t *features)
struct virtio_user_backend_ops virtio_ops_kernel = {
.setup = vhost_kernel_setup,
+ .destroy = vhost_kernel_destroy,
.get_backend_features = vhost_kernel_get_backend_features,
.set_owner = vhost_kernel_set_owner,
.get_features = vhost_kernel_get_features,
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index f67c40e047..e96b1d8b9c 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -17,6 +17,9 @@
#include "vhost.h"
#include "virtio_user_dev.h"
+struct vhost_user_data {
+ uint64_t protocol_features;
+};
#ifndef VHOST_USER_F_PROTOCOL_FEATURES
#define VHOST_USER_F_PROTOCOL_FEATURES 30
@@ -274,6 +277,7 @@ static int
vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_GET_FEATURES,
.flags = VHOST_USER_VERSION,
@@ -303,17 +307,17 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
return 0;
/* Negotiate protocol features */
- ret = vhost_user_get_protocol_features(dev, &dev->protocol_features);
+ ret = vhost_user_get_protocol_features(dev, &data->protocol_features);
if (ret < 0)
goto err;
- dev->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
+ data->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
- ret = vhost_user_set_protocol_features(dev, dev->protocol_features);
+ ret = vhost_user_set_protocol_features(dev, data->protocol_features);
if (ret < 0)
goto err;
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
+ if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
return 0;
@@ -429,12 +433,13 @@ vhost_user_set_memory_table(struct virtio_user_dev *dev)
struct walk_arg wa;
int fds[VHOST_MEMORY_MAX_NREGIONS];
int ret, fd_num;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_MEM_TABLE,
.flags = VHOST_USER_VERSION,
};
- if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
+ if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
msg.flags |= VHOST_USER_NEED_REPLY_MASK;
wa.region_nr = 0;
@@ -613,6 +618,7 @@ static int
vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_GET_STATUS,
.flags = VHOST_USER_VERSION,
@@ -629,7 +635,7 @@ vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
return -ENOTSUP;
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
+ if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
return -ENOTSUP;
ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
@@ -666,6 +672,7 @@ static int
vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_STATUS,
.flags = VHOST_USER_VERSION,
@@ -673,7 +680,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
.payload.u64 = status,
};
- if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
+ if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
msg.flags |= VHOST_USER_NEED_REPLY_MASK;
/*
@@ -687,7 +694,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
return -ENOTSUP;
- if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
+ if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
return -ENOTSUP;
ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
@@ -747,6 +754,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
int fd;
int flag;
struct sockaddr_un un;
+ struct vhost_user_data *data;
+
+ data = malloc(sizeof(*data));
+ if (!data) {
+ PMD_DRV_LOG(ERR, "(%s) Failed to allocate Vhost-user data\n", dev->path);
+ return -1;
+ }
+
+ memset(data, 0, sizeof(*data));
+
+ dev->backend_data = data;
fd = socket(AF_UNIX, SOCK_STREAM, 0);
if (fd < 0) {
@@ -781,6 +799,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
return 0;
}
+static int
+vhost_user_destroy(struct virtio_user_dev *dev)
+{
+ if (dev->backend_data) {
+ free(dev->backend_data);
+ dev->backend_data = NULL;
+ }
+
+ return 0;
+}
+
static int
vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
uint16_t pair_idx,
@@ -815,6 +844,7 @@ vhost_user_get_backend_features(uint64_t *features)
struct virtio_user_backend_ops virtio_ops_user = {
.setup = vhost_user_setup,
+ .destroy = vhost_user_destroy,
.get_backend_features = vhost_user_get_backend_features,
.set_owner = vhost_user_set_owner,
.get_features = vhost_user_get_features,
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index c826f333e0..b29426d767 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -287,6 +287,13 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
return 0;
}
+static int
+vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
+{
+ return;
+ return 0;
+}
+
static int
vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
uint16_t pair_idx,
@@ -322,6 +329,7 @@ vhost_vdpa_get_backend_features(uint64_t *features)
struct virtio_user_backend_ops virtio_ops_vdpa = {
.setup = vhost_vdpa_setup,
+ .destroy = vhost_vdpa_destroy,
.get_backend_features = vhost_vdpa_get_backend_features,
.set_owner = vhost_vdpa_set_owner,
.get_features = vhost_vdpa_get_features,
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 974de133bc..8d19a0addd 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -620,6 +620,8 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
if (dev->is_server)
unlink(dev->path);
+
+ dev->ops->destroy(dev);
}
uint8_t
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index b2da2944e9..ec73d5de11 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -30,7 +30,6 @@ struct virtio_user_dev {
int vhostfd;
int listenfd; /* listening fd */
bool is_server; /* server or client mode */
- uint64_t protocol_features; /* negotiated protocol features*/
/* for vhost_kernel backend */
char *ifname;
@@ -66,6 +65,8 @@ struct virtio_user_dev {
struct virtio_user_backend_ops *ops;
pthread_mutex_t mutex;
bool started;
+
+ void *backend_data;
};
int virtio_user_dev_set_features(struct virtio_user_dev *dev);
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data
2020-12-20 21:14 ` [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data Maxime Coquelin
@ 2021-01-05 21:26 ` David Marchand
2021-01-13 17:18 ` Adrian Moreno
1 sibling, 0 replies; 149+ messages in thread
From: David Marchand @ 2021-01-05 21:26 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, Xia, Chenbo, Olivier Matz, Adrian Moreno Zapata
On Sun, Dec 20, 2020 at 10:16 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 2fd00afa84..023fddcd69 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -357,6 +357,12 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_kernel_destroy(struct virtio_user_dev *dev)
> +{
Missing a __rte_unused.
> + return 0;
> +}
> +
> static int
> vhost_kernel_set_backend(int vhostfd, int tapfd)
> {
> @@ -455,6 +461,7 @@ vhost_kernel_get_backend_features(uint64_t *features)
>
> struct virtio_user_backend_ops virtio_ops_kernel = {
> .setup = vhost_kernel_setup,
> + .destroy = vhost_kernel_destroy,
> .get_backend_features = vhost_kernel_get_backend_features,
> .set_owner = vhost_kernel_set_owner,
> .get_features = vhost_kernel_get_features,
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index c826f333e0..b29426d767 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -287,6 +287,13 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
> +{
> + return;
> + return 0;
Rebase damage ? :)
> +}
> +
--
David Marchand
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data
2020-12-20 21:14 ` [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data Maxime Coquelin
2021-01-05 21:26 ` David Marchand
@ 2021-01-13 17:18 ` Adrian Moreno
2021-01-15 16:49 ` Maxime Coquelin
1 sibling, 1 reply; 149+ messages in thread
From: Adrian Moreno @ 2021-01-13 17:18 UTC (permalink / raw)
To: Maxime Coquelin, dev, chenbo.xia, olivier.matz, david.marchand
On 12/20/20 10:14 PM, Maxime Coquelin wrote:
> The goal of this patch is to introduce backend-specific
> data in order to better isolate what is backend-specific
> from what is generic to Virtio-user.
>
> For now, only Vhost-user protocol features are moved to
> Vhost-user backend data.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 1 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 7 +++
> drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++----
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++
> .../net/virtio/virtio_user/virtio_user_dev.c | 2 +
> .../net/virtio/virtio_user/virtio_user_dev.h | 3 +-
> 6 files changed, 58 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
> index b488ac78a1..ee5598226d 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -57,6 +57,7 @@ struct virtio_user_dev;
>
> struct virtio_user_backend_ops {
> int (*setup)(struct virtio_user_dev *dev);
> + int (*destroy)(struct virtio_user_dev *dev);
> int (*get_backend_features)(uint64_t *features);
> int (*set_owner)(struct virtio_user_dev *dev);
> int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 2fd00afa84..023fddcd69 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -357,6 +357,12 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_kernel_destroy(struct virtio_user_dev *dev)
> +{
> + return 0;
> +}
> +
> static int
> vhost_kernel_set_backend(int vhostfd, int tapfd)
> {
> @@ -455,6 +461,7 @@ vhost_kernel_get_backend_features(uint64_t *features)
>
> struct virtio_user_backend_ops virtio_ops_kernel = {
> .setup = vhost_kernel_setup,
> + .destroy = vhost_kernel_destroy,
> .get_backend_features = vhost_kernel_get_backend_features,
> .set_owner = vhost_kernel_set_owner,
> .get_features = vhost_kernel_get_features,
> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
> index f67c40e047..e96b1d8b9c 100644
> --- a/drivers/net/virtio/virtio_user/vhost_user.c
> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> @@ -17,6 +17,9 @@
> #include "vhost.h"
> #include "virtio_user_dev.h"
>
> +struct vhost_user_data {
> + uint64_t protocol_features;
> +};
>
> #ifndef VHOST_USER_F_PROTOCOL_FEATURES
> #define VHOST_USER_F_PROTOCOL_FEATURES 30
> @@ -274,6 +277,7 @@ static int
> vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
> {
> int ret;
> + struct vhost_user_data *data = dev->backend_data;
> struct vhost_user_msg msg = {
> .request = VHOST_USER_GET_FEATURES,
> .flags = VHOST_USER_VERSION,
> @@ -303,17 +307,17 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
> return 0;
>
> /* Negotiate protocol features */
> - ret = vhost_user_get_protocol_features(dev, &dev->protocol_features);
> + ret = vhost_user_get_protocol_features(dev, &data->protocol_features);
> if (ret < 0)
> goto err;
>
> - dev->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
> + data->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
>
> - ret = vhost_user_set_protocol_features(dev, dev->protocol_features);
> + ret = vhost_user_set_protocol_features(dev, data->protocol_features);
> if (ret < 0)
> goto err;
>
> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
> dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
>
> return 0;
> @@ -429,12 +433,13 @@ vhost_user_set_memory_table(struct virtio_user_dev *dev)
> struct walk_arg wa;
> int fds[VHOST_MEMORY_MAX_NREGIONS];
> int ret, fd_num;
> + struct vhost_user_data *data = dev->backend_data;
> struct vhost_user_msg msg = {
> .request = VHOST_USER_SET_MEM_TABLE,
> .flags = VHOST_USER_VERSION,
> };
>
> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> + if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>
> wa.region_nr = 0;
> @@ -613,6 +618,7 @@ static int
> vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
> {
> int ret;
> + struct vhost_user_data *data = dev->backend_data;
> struct vhost_user_msg msg = {
> .request = VHOST_USER_GET_STATUS,
> .flags = VHOST_USER_VERSION,
> @@ -629,7 +635,7 @@ vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
> if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
> return -ENOTSUP;
>
> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> return -ENOTSUP;
>
> ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> @@ -666,6 +672,7 @@ static int
> vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
> {
> int ret;
> + struct vhost_user_data *data = dev->backend_data;
> struct vhost_user_msg msg = {
> .request = VHOST_USER_SET_STATUS,
> .flags = VHOST_USER_VERSION,
> @@ -673,7 +680,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
> .payload.u64 = status,
> };
>
> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> + if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
> msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>
> /*
> @@ -687,7 +694,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
> if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
> return -ENOTSUP;
>
> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
> return -ENOTSUP;
>
> ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
> @@ -747,6 +754,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
> int fd;
> int flag;
> struct sockaddr_un un;
> + struct vhost_user_data *data;
> +
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_DRV_LOG(ERR, "(%s) Failed to allocate Vhost-user data\n", dev->path);
> + return -1;
> + }
> +
> + memset(data, 0, sizeof(*data));
> +
> + dev->backend_data = data;
>
> fd = socket(AF_UNIX, SOCK_STREAM, 0);
> if (fd < 0) {
There are a number of "return -1;" statements below which, if executed, would
now leak.
Also, there are two in virtio_user_dev.c:virtio_user_dev_setup()
> @@ -781,6 +799,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_user_destroy(struct virtio_user_dev *dev)
> +{
> + if (dev->backend_data) {
> + free(dev->backend_data);
> + dev->backend_data = NULL;
> + }
> +
> + return 0;
> +}
> +
> static int
> vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
> uint16_t pair_idx,
> @@ -815,6 +844,7 @@ vhost_user_get_backend_features(uint64_t *features)
>
> struct virtio_user_backend_ops virtio_ops_user = {
> .setup = vhost_user_setup,
> + .destroy = vhost_user_destroy,
> .get_backend_features = vhost_user_get_backend_features,
> .set_owner = vhost_user_set_owner,
> .get_features = vhost_user_get_features,
> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> index c826f333e0..b29426d767 100644
> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
> @@ -287,6 +287,13 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
> return 0;
> }
>
> +static int
> +vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
> +{
> + return;
> + return 0;
> +}
> +
> static int
> vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
> uint16_t pair_idx,
> @@ -322,6 +329,7 @@ vhost_vdpa_get_backend_features(uint64_t *features)
>
> struct virtio_user_backend_ops virtio_ops_vdpa = {
> .setup = vhost_vdpa_setup,
> + .destroy = vhost_vdpa_destroy,
> .get_backend_features = vhost_vdpa_get_backend_features,
> .set_owner = vhost_vdpa_set_owner,
> .get_features = vhost_vdpa_get_features,
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 974de133bc..8d19a0addd 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -620,6 +620,8 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
>
> if (dev->is_server)
> unlink(dev->path);
> +
> + dev->ops->destroy(dev);
> }
>
> uint8_t
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index b2da2944e9..ec73d5de11 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -30,7 +30,6 @@ struct virtio_user_dev {
> int vhostfd;
> int listenfd; /* listening fd */
> bool is_server; /* server or client mode */
> - uint64_t protocol_features; /* negotiated protocol features*/
>
> /* for vhost_kernel backend */
> char *ifname;
> @@ -66,6 +65,8 @@ struct virtio_user_dev {
> struct virtio_user_backend_ops *ops;
> pthread_mutex_t mutex;
> bool started;
> +
> + void *backend_data;
> };
>
> int virtio_user_dev_set_features(struct virtio_user_dev *dev);
>
--
Adrián Moreno
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data
2021-01-13 17:18 ` Adrian Moreno
@ 2021-01-15 16:49 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 16:49 UTC (permalink / raw)
To: Adrian Moreno, dev, chenbo.xia, olivier.matz, david.marchand
On 1/13/21 6:18 PM, Adrian Moreno wrote:
>
>
> On 12/20/20 10:14 PM, Maxime Coquelin wrote:
>> The goal of this patch is to introduce backend-specific
>> data in order to better isolate what is backend-specific
>> from what is generic to Virtio-user.
>>
>> For now, only Vhost-user protocol features are moved to
>> Vhost-user backend data.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 1 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 7 +++
>> drivers/net/virtio/virtio_user/vhost_user.c | 46 +++++++++++++++----
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 8 ++++
>> .../net/virtio/virtio_user/virtio_user_dev.c | 2 +
>> .../net/virtio/virtio_user/virtio_user_dev.h | 3 +-
>> 6 files changed, 58 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
>> index b488ac78a1..ee5598226d 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -57,6 +57,7 @@ struct virtio_user_dev;
>>
>> struct virtio_user_backend_ops {
>> int (*setup)(struct virtio_user_dev *dev);
>> + int (*destroy)(struct virtio_user_dev *dev);
>> int (*get_backend_features)(uint64_t *features);
>> int (*set_owner)(struct virtio_user_dev *dev);
>> int (*get_features)(struct virtio_user_dev *dev, uint64_t *features);
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 2fd00afa84..023fddcd69 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -357,6 +357,12 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
>> return 0;
>> }
>>
>> +static int
>> +vhost_kernel_destroy(struct virtio_user_dev *dev)
>> +{
>> + return 0;
>> +}
>> +
>> static int
>> vhost_kernel_set_backend(int vhostfd, int tapfd)
>> {
>> @@ -455,6 +461,7 @@ vhost_kernel_get_backend_features(uint64_t *features)
>>
>> struct virtio_user_backend_ops virtio_ops_kernel = {
>> .setup = vhost_kernel_setup,
>> + .destroy = vhost_kernel_destroy,
>> .get_backend_features = vhost_kernel_get_backend_features,
>> .set_owner = vhost_kernel_set_owner,
>> .get_features = vhost_kernel_get_features,
>> diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
>> index f67c40e047..e96b1d8b9c 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_user.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_user.c
>> @@ -17,6 +17,9 @@
>> #include "vhost.h"
>> #include "virtio_user_dev.h"
>>
>> +struct vhost_user_data {
>> + uint64_t protocol_features;
>> +};
>>
>> #ifndef VHOST_USER_F_PROTOCOL_FEATURES
>> #define VHOST_USER_F_PROTOCOL_FEATURES 30
>> @@ -274,6 +277,7 @@ static int
>> vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
>> {
>> int ret;
>> + struct vhost_user_data *data = dev->backend_data;
>> struct vhost_user_msg msg = {
>> .request = VHOST_USER_GET_FEATURES,
>> .flags = VHOST_USER_VERSION,
>> @@ -303,17 +307,17 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
>> return 0;
>>
>> /* Negotiate protocol features */
>> - ret = vhost_user_get_protocol_features(dev, &dev->protocol_features);
>> + ret = vhost_user_get_protocol_features(dev, &data->protocol_features);
>> if (ret < 0)
>> goto err;
>>
>> - dev->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
>> + data->protocol_features &= VHOST_USER_SUPPORTED_PROTOCOL_FEATURES;
>>
>> - ret = vhost_user_set_protocol_features(dev, dev->protocol_features);
>> + ret = vhost_user_set_protocol_features(dev, data->protocol_features);
>> if (ret < 0)
>> goto err;
>>
>> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
>> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)))
>> dev->unsupported_features |= (1ull << VIRTIO_NET_F_MQ);
>>
>> return 0;
>> @@ -429,12 +433,13 @@ vhost_user_set_memory_table(struct virtio_user_dev *dev)
>> struct walk_arg wa;
>> int fds[VHOST_MEMORY_MAX_NREGIONS];
>> int ret, fd_num;
>> + struct vhost_user_data *data = dev->backend_data;
>> struct vhost_user_msg msg = {
>> .request = VHOST_USER_SET_MEM_TABLE,
>> .flags = VHOST_USER_VERSION,
>> };
>>
>> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> + if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>>
>> wa.region_nr = 0;
>> @@ -613,6 +618,7 @@ static int
>> vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
>> {
>> int ret;
>> + struct vhost_user_data *data = dev->backend_data;
>> struct vhost_user_msg msg = {
>> .request = VHOST_USER_GET_STATUS,
>> .flags = VHOST_USER_VERSION,
>> @@ -629,7 +635,7 @@ vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
>> if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
>> return -ENOTSUP;
>>
>> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> return -ENOTSUP;
>>
>> ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
>> @@ -666,6 +672,7 @@ static int
>> vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
>> {
>> int ret;
>> + struct vhost_user_data *data = dev->backend_data;
>> struct vhost_user_msg msg = {
>> .request = VHOST_USER_SET_STATUS,
>> .flags = VHOST_USER_VERSION,
>> @@ -673,7 +680,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
>> .payload.u64 = status,
>> };
>>
>> - if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> + if (data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK))
>> msg.flags |= VHOST_USER_NEED_REPLY_MASK;
>>
>> /*
>> @@ -687,7 +694,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
>> if (!(dev->device_features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)))
>> return -ENOTSUP;
>>
>> - if (!(dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> + if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
>> return -ENOTSUP;
>>
>> ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
>> @@ -747,6 +754,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
>> int fd;
>> int flag;
>> struct sockaddr_un un;
>> + struct vhost_user_data *data;
>> +
>> + data = malloc(sizeof(*data));
>> + if (!data) {
>> + PMD_DRV_LOG(ERR, "(%s) Failed to allocate Vhost-user data\n", dev->path);
>> + return -1;
>> + }
>> +
>> + memset(data, 0, sizeof(*data));
>> +
>> + dev->backend_data = data;
>>
>> fd = socket(AF_UNIX, SOCK_STREAM, 0);
>> if (fd < 0) {
>
> There are a number of "return -1;" statements below which, if executed, would
> now leak.
Fixed.
> Also, there are two in virtio_user_dev.c:virtio_user_dev_setup()
Good catch, I'm adding a new patch to handle failure properly.
Also, I'll fix destroy callbacks not to crash if called twice or called
after setup callback failure.
>> @@ -781,6 +799,17 @@ vhost_user_setup(struct virtio_user_dev *dev)
>> return 0;
>> }
>>
>> +static int
>> +vhost_user_destroy(struct virtio_user_dev *dev)
>> +{
>> + if (dev->backend_data) {
>> + free(dev->backend_data);
>> + dev->backend_data = NULL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static int
>> vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
>> uint16_t pair_idx,
>> @@ -815,6 +844,7 @@ vhost_user_get_backend_features(uint64_t *features)
>>
>> struct virtio_user_backend_ops virtio_ops_user = {
>> .setup = vhost_user_setup,
>> + .destroy = vhost_user_destroy,
>> .get_backend_features = vhost_user_get_backend_features,
>> .set_owner = vhost_user_set_owner,
>> .get_features = vhost_user_get_features,
>> diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> index c826f333e0..b29426d767 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
>> @@ -287,6 +287,13 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
>> return 0;
>> }
>>
>> +static int
>> +vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
>> +{
>> + return;
>> + return 0;
>> +}
>> +
>> static int
>> vhost_vdpa_enable_queue_pair(struct virtio_user_dev *dev,
>> uint16_t pair_idx,
>> @@ -322,6 +329,7 @@ vhost_vdpa_get_backend_features(uint64_t *features)
>>
>> struct virtio_user_backend_ops virtio_ops_vdpa = {
>> .setup = vhost_vdpa_setup,
>> + .destroy = vhost_vdpa_destroy,
>> .get_backend_features = vhost_vdpa_get_backend_features,
>> .set_owner = vhost_vdpa_set_owner,
>> .get_features = vhost_vdpa_get_features,
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index 974de133bc..8d19a0addd 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -620,6 +620,8 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
>>
>> if (dev->is_server)
>> unlink(dev->path);
>> +
>> + dev->ops->destroy(dev);
>> }
>>
>> uint8_t
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> index b2da2944e9..ec73d5de11 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> @@ -30,7 +30,6 @@ struct virtio_user_dev {
>> int vhostfd;
>> int listenfd; /* listening fd */
>> bool is_server; /* server or client mode */
>> - uint64_t protocol_features; /* negotiated protocol features*/
>>
>> /* for vhost_kernel backend */
>> char *ifname;
>> @@ -66,6 +65,8 @@ struct virtio_user_dev {
>> struct virtio_user_backend_ops *ops;
>> pthread_mutex_t mutex;
>> bool started;
>> +
>> + void *backend_data;
>> };
>>
>> int virtio_user_dev_set_features(struct virtio_user_dev *dev);
>>
>
>
>
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (36 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 37/40] net/virtio: introduce backend data Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2021-01-07 6:32 ` Xia, Chenbo
2020-12-20 21:14 ` [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data " Maxime Coquelin
` (2 subsequent siblings)
40 siblings, 1 reply; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
This patch moves all the Vhost-user backend specific
logic like Vhost FD, listen FD and interrupt handling
to the vhost-user backend implementation.
In order to achieve that, new ops are created to update
the link status, disconnect and reconnect the server,
and fetch the link state interrupt FD.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost.h | 4 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 18 +-
drivers/net/virtio/virtio_user/vhost_user.c | 169 ++++++++++++++---
drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++
.../net/virtio/virtio_user/virtio_user_dev.c | 175 ++++++++++++++---
.../net/virtio/virtio_user/virtio_user_dev.h | 9 +-
drivers/net/virtio/virtio_user_ethdev.c | 179 +-----------------
7 files changed, 340 insertions(+), 230 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index ee5598226d..6001d7a164 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -75,6 +75,10 @@ struct virtio_user_backend_ops {
int (*enable_qp)(struct virtio_user_dev *dev, uint16_t pair_idx, int enable);
int (*dma_map)(struct virtio_user_dev *dev, void *addr, uint64_t iova, size_t len);
int (*dma_unmap)(struct virtio_user_dev *dev, void *addr, uint64_t iova, size_t len);
+ int (*update_link_state)(struct virtio_user_dev *dev);
+ int (*server_disconnect)(struct virtio_user_dev *dev);
+ int (*server_reconnect)(struct virtio_user_dev *dev);
+ int (*get_intr_fd)(struct virtio_user_dev *dev);
};
extern struct virtio_user_backend_ops virtio_ops_user;
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 023fddcd69..0ba37b23dc 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -459,6 +459,20 @@ vhost_kernel_get_backend_features(uint64_t *features)
return 0;
}
+static int
+vhost_kernel_update_link_state(struct virtio_user_dev *dev __rte_unused)
+{
+ /* Nothing to update (Maybe get TAP interface link state?) */
+ return 0;
+}
+
+static int
+vhost_kernel_get_intr_fd(struct virtio_user_dev *dev __rte_unused)
+{
+ /* No link state interrupt with Vhost-kernel */
+ return -1;
+}
+
struct virtio_user_backend_ops virtio_ops_kernel = {
.setup = vhost_kernel_setup,
.destroy = vhost_kernel_destroy,
@@ -475,5 +489,7 @@ struct virtio_user_backend_ops virtio_ops_kernel = {
.set_vring_addr = vhost_kernel_set_vring_addr,
.get_status = vhost_kernel_get_status,
.set_status = vhost_kernel_set_status,
- .enable_qp = vhost_kernel_enable_queue_pair
+ .enable_qp = vhost_kernel_enable_queue_pair,
+ .update_link_state = vhost_kernel_update_link_state,
+ .get_intr_fd = vhost_kernel_get_intr_fd,
};
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index e96b1d8b9c..9892135bce 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -11,6 +11,7 @@
#include <string.h>
#include <errno.h>
+#include <rte_alarm.h>
#include <rte_string_fns.h>
#include <rte_fbarray.h>
@@ -18,6 +19,8 @@
#include "virtio_user_dev.h"
struct vhost_user_data {
+ int vhostfd;
+ int listenfd;
uint64_t protocol_features;
};
@@ -179,13 +182,14 @@ vhost_user_read(int fd, struct vhost_user_msg *msg)
static int
vhost_user_check_reply_ack(struct virtio_user_dev *dev, struct vhost_user_msg *msg)
{
+ struct vhost_user_data *data = dev->backend_data;
enum vhost_user_request req = msg->request;
int ret;
if (!(msg->flags & VHOST_USER_NEED_REPLY_MASK))
return 0;
- ret = vhost_user_read(dev->vhostfd, msg);
+ ret = vhost_user_read(data->vhostfd, msg);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to read reply-ack");
return -1;
@@ -203,12 +207,13 @@ static int
vhost_user_set_owner(struct virtio_user_dev *dev)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_OWNER,
.flags = VHOST_USER_VERSION,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set owner");
return -1;
@@ -221,16 +226,17 @@ static int
vhost_user_get_protocol_features(struct virtio_user_dev *dev, uint64_t *features)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_GET_PROTOCOL_FEATURES,
.flags = VHOST_USER_VERSION,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0)
goto err;
- ret = vhost_user_read(dev->vhostfd, &msg);
+ ret = vhost_user_read(data->vhostfd, &msg);
if (ret < 0)
goto err;
@@ -257,6 +263,7 @@ static int
vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_PROTOCOL_FEATURES,
.flags = VHOST_USER_VERSION,
@@ -264,7 +271,7 @@ vhost_user_set_protocol_features(struct virtio_user_dev *dev, uint64_t features)
.payload.u64 = features,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set protocol features");
return -1;
@@ -283,11 +290,11 @@ vhost_user_get_features(struct virtio_user_dev *dev, uint64_t *features)
.flags = VHOST_USER_VERSION,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0)
goto err;
- ret = vhost_user_read(dev->vhostfd, &msg);
+ ret = vhost_user_read(data->vhostfd, &msg);
if (ret < 0)
goto err;
@@ -331,6 +338,7 @@ static int
vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_FEATURES,
.flags = VHOST_USER_VERSION,
@@ -338,7 +346,7 @@ vhost_user_set_features(struct virtio_user_dev *dev, uint64_t features)
.payload.u64 = features,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set features");
return -1;
@@ -462,7 +470,7 @@ vhost_user_set_memory_table(struct virtio_user_dev *dev)
msg.size += sizeof(msg.payload.memory.padding);
msg.size += fd_num * sizeof(struct vhost_memory_region);
- ret = vhost_user_write(dev->vhostfd, &msg, fds, fd_num);
+ ret = vhost_user_write(data->vhostfd, &msg, fds, fd_num);
if (ret < 0)
goto err;
@@ -477,6 +485,7 @@ vhost_user_set_vring(struct virtio_user_dev *dev, enum vhost_user_request req,
struct vhost_vring_state *state)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = req,
.flags = VHOST_USER_VERSION,
@@ -484,7 +493,7 @@ vhost_user_set_vring(struct virtio_user_dev *dev, enum vhost_user_request req,
.payload.state = *state,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set vring state (request %d)", req);
return -1;
@@ -516,6 +525,7 @@ vhost_user_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state
{
int ret;
struct vhost_user_msg msg;
+ struct vhost_user_data *data = dev->backend_data;
unsigned int index = state->index;
ret = vhost_user_set_vring(dev, VHOST_USER_GET_VRING_BASE, state);
@@ -524,7 +534,7 @@ vhost_user_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state
goto err;
}
- ret = vhost_user_read(dev->vhostfd, &msg);
+ ret = vhost_user_read(data->vhostfd, &msg);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to read reply");
goto err;
@@ -560,6 +570,7 @@ vhost_user_set_vring_file(struct virtio_user_dev *dev, enum vhost_user_request r
int ret;
int fd = file->fd;
int num_fd = 0;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = req,
.flags = VHOST_USER_VERSION,
@@ -572,7 +583,7 @@ vhost_user_set_vring_file(struct virtio_user_dev *dev, enum vhost_user_request r
else
msg.payload.u64 |= VHOST_USER_VRING_NOFD_MASK;
- ret = vhost_user_write(dev->vhostfd, &msg, &fd, num_fd);
+ ret = vhost_user_write(data->vhostfd, &msg, &fd, num_fd);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set vring file (request %d)", req);
return -1;
@@ -598,6 +609,7 @@ static int
vhost_user_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *addr)
{
int ret;
+ struct vhost_user_data *data = dev->backend_data;
struct vhost_user_msg msg = {
.request = VHOST_USER_SET_VRING_ADDR,
.flags = VHOST_USER_VERSION,
@@ -605,7 +617,7 @@ vhost_user_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *
.payload.addr = *addr,
};
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to send vring addresses");
return -1;
@@ -638,13 +650,13 @@ vhost_user_get_status(struct virtio_user_dev *dev, uint8_t *status)
if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
return -ENOTSUP;
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to send request");
goto err;
}
- ret = vhost_user_read(dev->vhostfd, &msg);
+ ret = vhost_user_read(data->vhostfd, &msg);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to recv request");
goto err;
@@ -697,7 +709,7 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
if (!(data->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_STATUS)))
return -ENOTSUP;
- ret = vhost_user_write(dev->vhostfd, &msg, NULL, 0);
+ ret = vhost_user_write(data->vhostfd, &msg, NULL, 0);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to send get status request");
return -1;
@@ -708,11 +720,12 @@ vhost_user_set_status(struct virtio_user_dev *dev, uint8_t status)
#define MAX_VIRTIO_USER_BACKLOG 1
static int
-virtio_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
+vhost_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
{
int ret;
int flag;
- int fd = dev->listenfd;
+ struct vhost_user_data *data = dev->backend_data;
+ int fd = data->listenfd;
ret = bind(fd, (struct sockaddr *)un, sizeof(*un));
if (ret < 0) {
@@ -725,8 +738,8 @@ virtio_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
return -1;
PMD_DRV_LOG(NOTICE, "(%s) waiting for client connection...", dev->path);
- dev->vhostfd = accept(fd, NULL, NULL);
- if (dev->vhostfd < 0) {
+ data->vhostfd = accept(fd, NULL, NULL);
+ if (data->vhostfd < 0) {
PMD_DRV_LOG(ERR, "Failed to accept initial client connection (%s)",
strerror(errno));
return -1;
@@ -741,6 +754,37 @@ virtio_user_start_server(struct virtio_user_dev *dev, struct sockaddr_un *un)
return 0;
}
+static int
+vhost_user_server_disconnect(struct virtio_user_dev *dev)
+{
+ struct vhost_user_data *data = dev->backend_data;
+
+ if (data->vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "(%s) Expected valid Vhost FD", dev->path);
+ return -1;
+ }
+
+ close(data->vhostfd);
+ data->vhostfd = -1;
+
+ return 0;
+}
+
+static int
+vhost_user_server_reconnect(struct virtio_user_dev *dev)
+{
+ struct vhost_user_data *data = dev->backend_data;
+ int fd;
+
+ fd = accept(data->listenfd, NULL, NULL);
+ if (fd < 0)
+ return -1;
+
+ data->vhostfd = fd;
+
+ return 0;
+}
+
/**
* Set up environment to talk with a vhost user backend.
*
@@ -766,6 +810,8 @@ vhost_user_setup(struct virtio_user_dev *dev)
dev->backend_data = data;
+ data->vhostfd = -1;
+
fd = socket(AF_UNIX, SOCK_STREAM, 0);
if (fd < 0) {
PMD_DRV_LOG(ERR, "socket() error, %s", strerror(errno));
@@ -781,8 +827,8 @@ vhost_user_setup(struct virtio_user_dev *dev)
strlcpy(un.sun_path, dev->path, sizeof(un.sun_path));
if (dev->is_server) {
- dev->listenfd = fd;
- if (virtio_user_start_server(dev, &un) < 0) {
+ data->listenfd = fd;
+ if (vhost_user_start_server(dev, &un) < 0) {
PMD_DRV_LOG(ERR, "virtio-user startup fails in server mode");
close(fd);
return -1;
@@ -793,7 +839,7 @@ vhost_user_setup(struct virtio_user_dev *dev)
close(fd);
return -1;
}
- dev->vhostfd = fd;
+ data->vhostfd = fd;
}
return 0;
@@ -802,6 +848,18 @@ vhost_user_setup(struct virtio_user_dev *dev)
static int
vhost_user_destroy(struct virtio_user_dev *dev)
{
+ struct vhost_user_data *data = dev->backend_data;
+
+ if (data->vhostfd >= 0) {
+ close(data->vhostfd);
+ data->vhostfd = -1;
+ }
+
+ if (data->listenfd >= 0) {
+ close(data->listenfd);
+ data->listenfd = -1;
+ }
+
if (dev->backend_data) {
free(dev->backend_data);
dev->backend_data = NULL;
@@ -815,8 +873,12 @@ vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
uint16_t pair_idx,
int enable)
{
+ struct vhost_user_data *data = dev->backend_data;
int i;
+ if (data->vhostfd < 0)
+ return 0;
+
if (dev->qp_enabled[pair_idx] == enable)
return 0;
@@ -842,6 +904,61 @@ vhost_user_get_backend_features(uint64_t *features)
return 0;
}
+static int
+vhost_user_update_link_state(struct virtio_user_dev *dev)
+{
+ struct vhost_user_data *data = dev->backend_data;
+ char buf[128];
+
+ if (data->vhostfd >= 0) {
+ int r;
+ int flags;
+
+ flags = fcntl(data->vhostfd, F_GETFL);
+ if (fcntl(data->vhostfd, F_SETFL, flags | O_NONBLOCK) == -1) {
+ PMD_DRV_LOG(ERR, "error setting O_NONBLOCK flag");
+ return -1;
+ }
+
+ r = recv(data->vhostfd, buf, 128, MSG_PEEK);
+ if (r == 0 || (r < 0 && errno != EAGAIN)) {
+ dev->net_status &= (~VIRTIO_NET_S_LINK_UP);
+ PMD_DRV_LOG(ERR, "virtio-user port %u is down", dev->port_id);
+
+ /* This function could be called in the process
+ * of interrupt handling, callback cannot be
+ * unregistered here, set an alarm to do it.
+ */
+ rte_eal_alarm_set(1, virtio_user_dev_delayed_handler, (void *)dev);
+ } else {
+ dev->net_status |= VIRTIO_NET_S_LINK_UP;
+ }
+
+ if (fcntl(data->vhostfd, F_SETFL,
+ flags & ~O_NONBLOCK) == -1) {
+ PMD_DRV_LOG(ERR, "error clearing O_NONBLOCK flag");
+ return -1;
+ }
+ } else if (dev->is_server) {
+ dev->net_status &= (~VIRTIO_NET_S_LINK_UP);
+ if (virtio_user_dev_server_reconnect(dev) >= 0)
+ dev->net_status |= VIRTIO_NET_S_LINK_UP;
+ }
+
+ return 0;
+}
+
+static int
+vhost_user_get_intr_fd(struct virtio_user_dev *dev)
+{
+ struct vhost_user_data *data = dev->backend_data;
+
+ if (dev->is_server && data->vhostfd == -1)
+ return data->listenfd;
+
+ return data->vhostfd;
+}
+
struct virtio_user_backend_ops virtio_ops_user = {
.setup = vhost_user_setup,
.destroy = vhost_user_destroy,
@@ -859,5 +976,9 @@ struct virtio_user_backend_ops virtio_ops_user = {
.set_vring_addr = vhost_user_set_vring_addr,
.get_status = vhost_user_get_status,
.set_status = vhost_user_set_status,
- .enable_qp = vhost_user_enable_queue_pair
+ .enable_qp = vhost_user_enable_queue_pair,
+ .update_link_state = vhost_user_update_link_state,
+ .server_disconnect = vhost_user_server_disconnect,
+ .server_reconnect = vhost_user_server_reconnect,
+ .get_intr_fd = vhost_user_get_intr_fd,
};
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index b29426d767..0f422ae84a 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -327,6 +327,20 @@ vhost_vdpa_get_backend_features(uint64_t *features)
return 0;
}
+static int
+vhost_vdpa_update_link_state(struct virtio_user_dev *dev __rte_unused)
+{
+ /* Nothing to update (for now?) */
+ return 0;
+}
+
+static int
+vhost_vdpa_get_intr_fd(struct virtio_user_dev *dev __rte_unused)
+{
+ /* No link state interrupt with Vhost-vDPA */
+ return -1;
+}
+
struct virtio_user_backend_ops virtio_ops_vdpa = {
.setup = vhost_vdpa_setup,
.destroy = vhost_vdpa_destroy,
@@ -347,4 +361,6 @@ struct virtio_user_backend_ops virtio_ops_vdpa = {
.enable_qp = vhost_vdpa_enable_queue_pair,
.dma_map = vhost_vdpa_dma_map,
.dma_unmap = vhost_vdpa_dma_unmap,
+ .update_link_state = vhost_vdpa_update_link_state,
+ .get_intr_fd = vhost_vdpa_get_intr_fd,
};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 8d19a0addd..93a9ce2cd2 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -342,11 +342,7 @@ virtio_user_fill_intr_handle(struct virtio_user_dev *dev)
eth_dev->intr_handle->type = RTE_INTR_HANDLE_VDEV;
/* For virtio vdev, no need to read counter for clean */
eth_dev->intr_handle->efd_counter_size = 0;
- eth_dev->intr_handle->fd = -1;
- if (dev->vhostfd >= 0)
- eth_dev->intr_handle->fd = dev->vhostfd;
- else if (dev->is_server)
- eth_dev->intr_handle->fd = dev->listenfd;
+ eth_dev->intr_handle->fd = dev->ops->get_intr_fd(dev);
return 0;
}
@@ -403,7 +399,6 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
{
uint32_t q;
- dev->vhostfd = -1;
dev->vhostfds = NULL;
dev->tapfds = NULL;
@@ -597,15 +592,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
close(dev->callfds[i]);
close(dev->kickfds[i]);
}
-
- if (dev->vhostfd >= 0)
- close(dev->vhostfd);
-
- if (dev->is_server && dev->listenfd >= 0) {
- close(dev->listenfd);
- dev->listenfd = -1;
- }
-
if (dev->vhostfds) {
for (i = 0; i < dev->max_queue_pairs; ++i) {
close(dev->vhostfds[i]);
@@ -636,15 +622,11 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
return -1;
}
- /* Server mode can't enable queue pairs if vhostfd is invalid,
- * always return 0 in this case.
- */
- if (!dev->is_server || dev->vhostfd >= 0) {
- for (i = 0; i < q_pairs; ++i)
- ret |= dev->ops->enable_qp(dev, i, 1);
- for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= dev->ops->enable_qp(dev, i, 0);
- }
+ for (i = 0; i < q_pairs; ++i)
+ ret |= dev->ops->enable_qp(dev, i, 1);
+ for (i = q_pairs; i < dev->max_queue_pairs; ++i)
+ ret |= dev->ops->enable_qp(dev, i, 0);
+
dev->queue_pairs = q_pairs;
return ret;
@@ -859,3 +841,148 @@ virtio_user_dev_update_status(struct virtio_user_dev *dev)
pthread_mutex_unlock(&dev->mutex);
return ret;
}
+
+int
+virtio_user_dev_update_link_state(struct virtio_user_dev *dev)
+{
+ if (dev->ops->update_link_state)
+ return dev->ops->update_link_state(dev);
+
+ return 0;
+}
+
+static void
+virtio_user_dev_reset_queues_packed(struct rte_eth_dev *eth_dev)
+{
+ struct virtio_user_dev *dev = eth_dev->data->dev_private;
+ struct virtio_hw *hw = &dev->hw;
+ struct virtnet_rx *rxvq;
+ struct virtnet_tx *txvq;
+ uint16_t i;
+
+ /* Add lock to avoid queue contention. */
+ rte_spinlock_lock(&hw->state_lock);
+ hw->started = 0;
+
+ /*
+ * Waitting for datapath to complete before resetting queues.
+ * 1 ms should be enough for the ongoing Tx/Rx function to finish.
+ */
+ rte_delay_ms(1);
+
+ /* Vring reset for each Tx queue and Rx queue. */
+ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
+ rxvq = eth_dev->data->rx_queues[i];
+ virtqueue_rxvq_reset_packed(rxvq->vq);
+ virtio_dev_rx_queue_setup_finish(eth_dev, i);
+ }
+
+ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
+ txvq = eth_dev->data->tx_queues[i];
+ virtqueue_txvq_reset_packed(txvq->vq);
+ }
+
+ hw->started = 1;
+ rte_spinlock_unlock(&hw->state_lock);
+}
+
+void
+virtio_user_dev_delayed_handler(void *param)
+{
+ struct virtio_user_dev *dev = param;
+ struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
+
+ if (rte_intr_disable(eth_dev->intr_handle) < 0) {
+ PMD_DRV_LOG(ERR, "interrupt disable failed");
+ return;
+ }
+ rte_intr_callback_unregister(eth_dev->intr_handle,
+ virtio_interrupt_handler, eth_dev);
+ if (dev->is_server) {
+ dev->ops->server_disconnect(dev);
+ eth_dev->intr_handle->fd = dev->ops->get_intr_fd(dev);
+ rte_intr_callback_register(eth_dev->intr_handle,
+ virtio_interrupt_handler, eth_dev);
+ if (rte_intr_enable(eth_dev->intr_handle) < 0) {
+ PMD_DRV_LOG(ERR, "interrupt enable failed");
+ return;
+ }
+ }
+}
+
+int
+virtio_user_dev_server_reconnect(struct virtio_user_dev *dev)
+{
+ int ret, old_status;
+ struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
+ struct virtio_hw *hw = &dev->hw;
+
+ if (!dev->ops->server_reconnect) {
+ PMD_DRV_LOG(ERR, "(%s) Missing server reconnect callback", dev->path);
+ return -1;
+ }
+
+ old_status = virtio_get_status(hw);
+
+ virtio_reset(hw);
+
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
+
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
+
+ if (dev->ops->get_features(dev, &dev->device_features) < 0) {
+ PMD_INIT_LOG(ERR, "get_features failed: %s",
+ strerror(errno));
+ return -1;
+ }
+
+ dev->device_features |= dev->frontend_features;
+
+ /* umask vhost-user unsupported features */
+ dev->device_features &= ~(dev->unsupported_features);
+
+ dev->features &= dev->device_features;
+
+ /* For packed ring, resetting queues is required in reconnection. */
+ if (virtio_with_packed_queue(hw) &&
+ (old_status & VIRTIO_CONFIG_STATUS_DRIVER_OK)) {
+ PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped"
+ " when packed ring reconnecting.");
+ virtio_user_dev_reset_queues_packed(eth_dev);
+ }
+
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
+
+ /* Start the device */
+ virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
+ if (!dev->started)
+ return -1;
+
+ if (dev->queue_pairs > 1) {
+ ret = virtio_user_handle_mq(dev, dev->queue_pairs);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Fails to enable multi-queue pairs!");
+ return -1;
+ }
+ }
+ if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) {
+ if (rte_intr_disable(eth_dev->intr_handle) < 0) {
+ PMD_DRV_LOG(ERR, "interrupt disable failed");
+ return -1;
+ }
+ rte_intr_callback_unregister(eth_dev->intr_handle,
+ virtio_interrupt_handler,
+ eth_dev);
+
+ eth_dev->intr_handle->fd = dev->ops->get_intr_fd(dev);
+ rte_intr_callback_register(eth_dev->intr_handle,
+ virtio_interrupt_handler, eth_dev);
+
+ if (rte_intr_enable(eth_dev->intr_handle) < 0) {
+ PMD_DRV_LOG(ERR, "interrupt enable failed");
+ return -1;
+ }
+ }
+ PMD_INIT_LOG(NOTICE, "server mode virtio-user reconnection succeeds!");
+ return 0;
+}
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index ec73d5de11..a429dcc57c 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -26,11 +26,11 @@ struct virtio_user_queue {
struct virtio_user_dev {
struct virtio_hw hw;
enum virtio_user_backend_type backend_type;
- /* for vhost_user backend */
- int vhostfd;
- int listenfd; /* listening fd */
bool is_server; /* server or client mode */
+ /* for vhost_vdpa backend */
+ int vhostfd;
+
/* for vhost_kernel backend */
char *ifname;
int *vhostfds;
@@ -84,5 +84,8 @@ void virtio_user_handle_cq_packed(struct virtio_user_dev *dev,
uint8_t virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs);
int virtio_user_dev_set_status(struct virtio_user_dev *dev, uint8_t status);
int virtio_user_dev_update_status(struct virtio_user_dev *dev);
+int virtio_user_dev_update_link_state(struct virtio_user_dev *dev);
+void virtio_user_dev_delayed_handler(void *param);
+int virtio_user_dev_server_reconnect(struct virtio_user_dev *dev);
extern const char * const virtio_user_backend_strings[];
#endif
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 77ad468768..a287805cf2 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -28,146 +28,6 @@
#define virtio_user_get_dev(hw) container_of(hw, struct virtio_user_dev, hw)
-static void
-virtio_user_reset_queues_packed(struct rte_eth_dev *eth_dev)
-{
- struct virtio_user_dev *dev = eth_dev->data->dev_private;
- struct virtio_hw *hw = &dev->hw;
- struct virtnet_rx *rxvq;
- struct virtnet_tx *txvq;
- uint16_t i;
-
- /* Add lock to avoid queue contention. */
- rte_spinlock_lock(&hw->state_lock);
- hw->started = 0;
-
- /*
- * Waitting for datapath to complete before resetting queues.
- * 1 ms should be enough for the ongoing Tx/Rx function to finish.
- */
- rte_delay_ms(1);
-
- /* Vring reset for each Tx queue and Rx queue. */
- for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
- rxvq = eth_dev->data->rx_queues[i];
- virtqueue_rxvq_reset_packed(rxvq->vq);
- virtio_dev_rx_queue_setup_finish(eth_dev, i);
- }
-
- for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
- txvq = eth_dev->data->tx_queues[i];
- virtqueue_txvq_reset_packed(txvq->vq);
- }
-
- hw->started = 1;
- rte_spinlock_unlock(&hw->state_lock);
-}
-
-
-static int
-virtio_user_server_reconnect(struct virtio_user_dev *dev)
-{
- int ret, connectfd, old_status;
- struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
- struct virtio_hw *hw = &dev->hw;
-
- connectfd = accept(dev->listenfd, NULL, NULL);
- if (connectfd < 0)
- return -1;
-
- dev->vhostfd = connectfd;
- old_status = virtio_get_status(hw);
-
- virtio_reset(hw);
-
- virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
-
- virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
-
- if (dev->ops->get_features(dev, &dev->device_features) < 0) {
- PMD_INIT_LOG(ERR, "get_features failed: %s",
- strerror(errno));
- return -1;
- }
-
- dev->device_features |= dev->frontend_features;
-
- /* umask vhost-user unsupported features */
- dev->device_features &= ~(dev->unsupported_features);
-
- dev->features &= dev->device_features;
-
- /* For packed ring, resetting queues is required in reconnection. */
- if (virtio_with_packed_queue(hw) &&
- (old_status & VIRTIO_CONFIG_STATUS_DRIVER_OK)) {
- PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped"
- " when packed ring reconnecting.");
- virtio_user_reset_queues_packed(eth_dev);
- }
-
- virtio_set_status(hw, VIRTIO_CONFIG_STATUS_FEATURES_OK);
-
- /* Start the device */
- virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER_OK);
- if (!dev->started)
- return -1;
-
- if (dev->queue_pairs > 1) {
- ret = virtio_user_handle_mq(dev, dev->queue_pairs);
- if (ret != 0) {
- PMD_INIT_LOG(ERR, "Fails to enable multi-queue pairs!");
- return -1;
- }
- }
- if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) {
- if (rte_intr_disable(eth_dev->intr_handle) < 0) {
- PMD_DRV_LOG(ERR, "interrupt disable failed");
- return -1;
- }
- rte_intr_callback_unregister(eth_dev->intr_handle,
- virtio_interrupt_handler,
- eth_dev);
- eth_dev->intr_handle->fd = connectfd;
- rte_intr_callback_register(eth_dev->intr_handle,
- virtio_interrupt_handler, eth_dev);
-
- if (rte_intr_enable(eth_dev->intr_handle) < 0) {
- PMD_DRV_LOG(ERR, "interrupt enable failed");
- return -1;
- }
- }
- PMD_INIT_LOG(NOTICE, "server mode virtio-user reconnection succeeds!");
- return 0;
-}
-
-static void
-virtio_user_delayed_handler(void *param)
-{
- struct virtio_hw *hw = (struct virtio_hw *)param;
- struct rte_eth_dev *eth_dev = &rte_eth_devices[hw->port_id];
- struct virtio_user_dev *dev = virtio_user_get_dev(hw);
-
- if (rte_intr_disable(eth_dev->intr_handle) < 0) {
- PMD_DRV_LOG(ERR, "interrupt disable failed");
- return;
- }
- rte_intr_callback_unregister(eth_dev->intr_handle,
- virtio_interrupt_handler, eth_dev);
- if (dev->is_server) {
- if (dev->vhostfd >= 0) {
- close(dev->vhostfd);
- dev->vhostfd = -1;
- }
- eth_dev->intr_handle->fd = dev->listenfd;
- rte_intr_callback_register(eth_dev->intr_handle,
- virtio_interrupt_handler, eth_dev);
- if (rte_intr_enable(eth_dev->intr_handle) < 0) {
- PMD_DRV_LOG(ERR, "interrupt enable failed");
- return;
- }
- }
-}
-
static void
virtio_user_read_dev_config(struct virtio_hw *hw, size_t offset,
void *dst, int length)
@@ -183,44 +43,7 @@ virtio_user_read_dev_config(struct virtio_hw *hw, size_t offset,
}
if (offset == offsetof(struct virtio_net_config, status)) {
- char buf[128];
-
- if (dev->vhostfd >= 0) {
- int r;
- int flags;
-
- flags = fcntl(dev->vhostfd, F_GETFL);
- if (fcntl(dev->vhostfd, F_SETFL,
- flags | O_NONBLOCK) == -1) {
- PMD_DRV_LOG(ERR, "error setting O_NONBLOCK flag");
- return;
- }
- r = recv(dev->vhostfd, buf, 128, MSG_PEEK);
- if (r == 0 || (r < 0 && errno != EAGAIN)) {
- dev->net_status &= (~VIRTIO_NET_S_LINK_UP);
- PMD_DRV_LOG(ERR, "virtio-user port %u is down",
- hw->port_id);
-
- /* This function could be called in the process
- * of interrupt handling, callback cannot be
- * unregistered here, set an alarm to do it.
- */
- rte_eal_alarm_set(1,
- virtio_user_delayed_handler,
- (void *)hw);
- } else {
- dev->net_status |= VIRTIO_NET_S_LINK_UP;
- }
- if (fcntl(dev->vhostfd, F_SETFL,
- flags & ~O_NONBLOCK) == -1) {
- PMD_DRV_LOG(ERR, "error clearing O_NONBLOCK flag");
- return;
- }
- } else if (dev->is_server) {
- dev->net_status &= (~VIRTIO_NET_S_LINK_UP);
- if (virtio_user_server_reconnect(dev) >= 0)
- dev->net_status |= VIRTIO_NET_S_LINK_UP;
- }
+ virtio_user_dev_update_link_state(dev);
*(uint16_t *)dst = dev->net_status;
}
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend Maxime Coquelin
@ 2021-01-07 6:32 ` Xia, Chenbo
2021-01-15 12:03 ` Maxime Coquelin
0 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-07 6:32 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend
>
> This patch moves all the Vhost-user backend specific
> logic like Vhost FD, listen FD and interrupt handling
> to the vhost-user backend implementation.
>
> In order to achieve that, new ops are created to update
> the link status, disconnect and reconnect the server,
> and fetch the link state interrupt FD.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost.h | 4 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 18 +-
> drivers/net/virtio/virtio_user/vhost_user.c | 169 ++++++++++++++---
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++
> .../net/virtio/virtio_user/virtio_user_dev.c | 175 ++++++++++++++---
> .../net/virtio/virtio_user/virtio_user_dev.h | 9 +-
> drivers/net/virtio/virtio_user_ethdev.c | 179 +-----------------
> 7 files changed, 340 insertions(+), 230 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost.h
> b/drivers/net/virtio/virtio_user/vhost.h
> index ee5598226d..6001d7a164 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -75,6 +75,10 @@ struct virtio_user_backend_ops {
> int (*enable_qp)(struct virtio_user_dev *dev, uint16_t pair_idx, int
> enable);
> int (*dma_map)(struct virtio_user_dev *dev, void *addr, uint64_t iova,
> size_t len);
> int (*dma_unmap)(struct virtio_user_dev *dev, void *addr, uint64_t iova,
> size_t len);
> + int (*update_link_state)(struct virtio_user_dev *dev);
> + int (*server_disconnect)(struct virtio_user_dev *dev);
> + int (*server_reconnect)(struct virtio_user_dev *dev);
> + int (*get_intr_fd)(struct virtio_user_dev *dev);
This get_intr_fd is for getting link state interrupt fd. I think his name is a little bit
generic. What do you think? Maybe we could change the name or add comment here to clarify here?
> };
>
> extern struct virtio_user_backend_ops virtio_ops_user;
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
> b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 023fddcd69..0ba37b23dc 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -459,6 +459,20 @@ vhost_kernel_get_backend_features(uint64_t *features)
> return 0;
> }
>
<snip>
> +static void
> +virtio_user_dev_reset_queues_packed(struct rte_eth_dev *eth_dev)
> +{
> + struct virtio_user_dev *dev = eth_dev->data->dev_private;
> + struct virtio_hw *hw = &dev->hw;
> + struct virtnet_rx *rxvq;
> + struct virtnet_tx *txvq;
> + uint16_t i;
> +
> + /* Add lock to avoid queue contention. */
> + rte_spinlock_lock(&hw->state_lock);
> + hw->started = 0;
> +
> + /*
> + * Waitting for datapath to complete before resetting queues.
s/Waitting/Waiting
> + * 1 ms should be enough for the ongoing Tx/Rx function to finish.
> + */
> + rte_delay_ms(1);
> +
> + /* Vring reset for each Tx queue and Rx queue. */
> + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
> + rxvq = eth_dev->data->rx_queues[i];
> + virtqueue_rxvq_reset_packed(rxvq->vq);
> + virtio_dev_rx_queue_setup_finish(eth_dev, i);
> + }
> +
> + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
> + txvq = eth_dev->data->tx_queues[i];
> + virtqueue_txvq_reset_packed(txvq->vq);
> + }
> +
> + hw->started = 1;
> + rte_spinlock_unlock(&hw->state_lock);
> +}
> +
> +void
> +virtio_user_dev_delayed_handler(void *param)
> +{
> + struct virtio_user_dev *dev = param;
> + struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
> +
> + if (rte_intr_disable(eth_dev->intr_handle) < 0) {
> + PMD_DRV_LOG(ERR, "interrupt disable failed");
> + return;
> + }
> + rte_intr_callback_unregister(eth_dev->intr_handle,
> + virtio_interrupt_handler, eth_dev);
> + if (dev->is_server) {
> + dev->ops->server_disconnect(dev);
Better to check ops existence too?
> + eth_dev->intr_handle->fd = dev->ops->get_intr_fd(dev);
> + rte_intr_callback_register(eth_dev->intr_handle,
> + virtio_interrupt_handler, eth_dev);
> + if (rte_intr_enable(eth_dev->intr_handle) < 0) {
> + PMD_DRV_LOG(ERR, "interrupt enable failed");
> + return;
> + }
> + }
> +}
> +
> +int
> +virtio_user_dev_server_reconnect(struct virtio_user_dev *dev)
> +{
> + int ret, old_status;
> + struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
> + struct virtio_hw *hw = &dev->hw;
> +
> + if (!dev->ops->server_reconnect) {
> + PMD_DRV_LOG(ERR, "(%s) Missing server reconnect callback", dev-
> >path);
> + return -1;
> + }
Forget to call server_reconnect? :)
> +
> + old_status = virtio_get_status(hw);
> +
> + virtio_reset(hw);
> +
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
> +
> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
> +
> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
> + PMD_INIT_LOG(ERR, "get_features failed: %s",
> + strerror(errno));
> + return -1;
> + }
> +
> + dev->device_features |= dev->frontend_features;
> +
> + /* umask vhost-user unsupported features */
s/umask/unmask
Thanks,
Chenbo
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend
2021-01-07 6:32 ` Xia, Chenbo
@ 2021-01-15 12:03 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 12:03 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/7/21 7:32 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend
>>
>> This patch moves all the Vhost-user backend specific
>> logic like Vhost FD, listen FD and interrupt handling
>> to the vhost-user backend implementation.
>>
>> In order to achieve that, new ops are created to update
>> the link status, disconnect and reconnect the server,
>> and fetch the link state interrupt FD.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost.h | 4 +
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 18 +-
>> drivers/net/virtio/virtio_user/vhost_user.c | 169 ++++++++++++++---
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 ++
>> .../net/virtio/virtio_user/virtio_user_dev.c | 175 ++++++++++++++---
>> .../net/virtio/virtio_user/virtio_user_dev.h | 9 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 179 +-----------------
>> 7 files changed, 340 insertions(+), 230 deletions(-)
>>
>> diff --git a/drivers/net/virtio/virtio_user/vhost.h
>> b/drivers/net/virtio/virtio_user/vhost.h
>> index ee5598226d..6001d7a164 100644
>> --- a/drivers/net/virtio/virtio_user/vhost.h
>> +++ b/drivers/net/virtio/virtio_user/vhost.h
>> @@ -75,6 +75,10 @@ struct virtio_user_backend_ops {
>> int (*enable_qp)(struct virtio_user_dev *dev, uint16_t pair_idx, int
>> enable);
>> int (*dma_map)(struct virtio_user_dev *dev, void *addr, uint64_t iova,
>> size_t len);
>> int (*dma_unmap)(struct virtio_user_dev *dev, void *addr, uint64_t iova,
>> size_t len);
>> + int (*update_link_state)(struct virtio_user_dev *dev);
>> + int (*server_disconnect)(struct virtio_user_dev *dev);
>> + int (*server_reconnect)(struct virtio_user_dev *dev);
>> + int (*get_intr_fd)(struct virtio_user_dev *dev);
>
> This get_intr_fd is for getting link state interrupt fd. I think his name is a little bit
> generic. What do you think? Maybe we could change the name or add comment here to clarify here?
>
>> };
>>
>> extern struct virtio_user_backend_ops virtio_ops_user;
>> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> index 023fddcd69..0ba37b23dc 100644
>> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
>> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
>> @@ -459,6 +459,20 @@ vhost_kernel_get_backend_features(uint64_t *features)
>> return 0;
>> }
>>
>
> <snip>
>
>> +static void
>> +virtio_user_dev_reset_queues_packed(struct rte_eth_dev *eth_dev)
>> +{
>> + struct virtio_user_dev *dev = eth_dev->data->dev_private;
>> + struct virtio_hw *hw = &dev->hw;
>> + struct virtnet_rx *rxvq;
>> + struct virtnet_tx *txvq;
>> + uint16_t i;
>> +
>> + /* Add lock to avoid queue contention. */
>> + rte_spinlock_lock(&hw->state_lock);
>> + hw->started = 0;
>> +
>> + /*
>> + * Waitting for datapath to complete before resetting queues.
>
> s/Waitting/Waiting
Fixed
>> + * 1 ms should be enough for the ongoing Tx/Rx function to finish.
>> + */
>> + rte_delay_ms(1);
>> +
>> + /* Vring reset for each Tx queue and Rx queue. */
>> + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
>> + rxvq = eth_dev->data->rx_queues[i];
>> + virtqueue_rxvq_reset_packed(rxvq->vq);
>> + virtio_dev_rx_queue_setup_finish(eth_dev, i);
>> + }
>> +
>> + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
>> + txvq = eth_dev->data->tx_queues[i];
>> + virtqueue_txvq_reset_packed(txvq->vq);
>> + }
>> +
>> + hw->started = 1;
>> + rte_spinlock_unlock(&hw->state_lock);
>> +}
>> +
>> +void
>> +virtio_user_dev_delayed_handler(void *param)
>> +{
>> + struct virtio_user_dev *dev = param;
>> + struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
>> +
>> + if (rte_intr_disable(eth_dev->intr_handle) < 0) {
>> + PMD_DRV_LOG(ERR, "interrupt disable failed");
>> + return;
>> + }
>> + rte_intr_callback_unregister(eth_dev->intr_handle,
>> + virtio_interrupt_handler, eth_dev);
>> + if (dev->is_server) {
>> + dev->ops->server_disconnect(dev);
>
> Better to check ops existence too?
Indeed, v2 will check its existence before calling.
>> + eth_dev->intr_handle->fd = dev->ops->get_intr_fd(dev);
>> + rte_intr_callback_register(eth_dev->intr_handle,
>> + virtio_interrupt_handler, eth_dev);
>> + if (rte_intr_enable(eth_dev->intr_handle) < 0) {
>> + PMD_DRV_LOG(ERR, "interrupt enable failed");
>> + return;
>> + }
>> + }
>> +}
>> +
>> +int
>> +virtio_user_dev_server_reconnect(struct virtio_user_dev *dev)
>> +{
>> + int ret, old_status;
>> + struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
>> + struct virtio_hw *hw = &dev->hw;
>> +
>> + if (!dev->ops->server_reconnect) {
>> + PMD_DRV_LOG(ERR, "(%s) Missing server reconnect callback", dev-
>>> path);
>> + return -1;
>> + }
>
> Forget to call server_reconnect? :)
Indeed! When I said I didn't tried last 5 patches, I wasn't lying! :)
>> +
>> + old_status = virtio_get_status(hw);
>> +
>> + virtio_reset(hw);
>> +
>> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_ACK);
>> +
>> + virtio_set_status(hw, VIRTIO_CONFIG_STATUS_DRIVER);
>> +
>> + if (dev->ops->get_features(dev, &dev->device_features) < 0) {
>> + PMD_INIT_LOG(ERR, "get_features failed: %s",
>> + strerror(errno));
>> + return -1;
>> + }
>> +
>> + dev->device_features |= dev->frontend_features;
>> +
>> + /* umask vhost-user unsupported features */
>
> s/umask/unmask
Fixed.
Thanks,
Maxime
> Thanks,
> Chenbo
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (37 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 38/40] net/virtio: move Vhost-user specifics to its backend Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2021-01-07 6:42 ` Xia, Chenbo
2021-01-11 8:02 ` Xia, Chenbo
2020-12-20 21:14 ` [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA " Maxime Coquelin
2020-12-21 10:58 ` [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
40 siblings, 2 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
As done earlier for Vhost-user, this patch moves the
Vhost-Kernel specific data to its backend file.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++----
.../net/virtio/virtio_user/virtio_user_dev.c | 43 ++--------
.../net/virtio/virtio_user/virtio_user_dev.h | 7 +-
3 files changed, 72 insertions(+), 56 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 0ba37b23dc..b8f5391733 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -14,6 +14,11 @@
#include "virtio_user_dev.h"
#include "vhost_kernel_tap.h"
+struct vhost_kernel_data {
+ int *vhostfds;
+ int *tapfds;
+};
+
struct vhost_memory_kernel {
uint32_t nregions;
uint32_t padding;
@@ -96,7 +101,9 @@ vhost_kernel_ioctl(int fd, uint64_t request, void *arg)
static int
vhost_kernel_set_owner(struct virtio_user_dev *dev)
{
- return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_OWNER, NULL);
+ struct vhost_kernel_data *data = dev->backend_data;
+
+ return vhost_kernel_ioctl(data->vhostfds[0], VHOST_SET_OWNER, NULL);
}
static int
@@ -104,8 +111,9 @@ vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
{
int ret;
unsigned int tap_features;
+ struct vhost_kernel_data *data = dev->backend_data;
- ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_GET_FEATURES, features);
+ ret = vhost_kernel_ioctl(data->vhostfds[0], VHOST_GET_FEATURES, features);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to get features");
return -1;
@@ -138,6 +146,8 @@ vhost_kernel_get_features(struct virtio_user_dev *dev, uint64_t *features)
static int
vhost_kernel_set_features(struct virtio_user_dev *dev, uint64_t features)
{
+ struct vhost_kernel_data *data = dev->backend_data;
+
/* We don't need memory protection here */
features &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
/* VHOST kernel does not know about below flags */
@@ -145,7 +155,7 @@ vhost_kernel_set_features(struct virtio_user_dev *dev, uint64_t features)
features &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
features &= ~(1ULL << VIRTIO_NET_F_MQ);
- return vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_FEATURES, &features);
+ return vhost_kernel_ioctl(data->vhostfds[0], VHOST_SET_FEATURES, &features);
}
static int
@@ -185,6 +195,7 @@ add_memseg_list(const struct rte_memseg_list *msl, void *arg)
static int
vhost_kernel_set_memory_table(struct virtio_user_dev *dev)
{
+ struct vhost_kernel_data *data = dev->backend_data;
struct vhost_memory_kernel *vm;
int ret;
@@ -205,7 +216,7 @@ vhost_kernel_set_memory_table(struct virtio_user_dev *dev)
if (ret < 0)
goto err_free;
- ret = vhost_kernel_ioctl(dev->vhostfds[0], VHOST_SET_MEM_TABLE, vm);
+ ret = vhost_kernel_ioctl(data->vhostfds[0], VHOST_SET_MEM_TABLE, vm);
if (ret < 0)
goto err_free;
@@ -224,9 +235,10 @@ vhost_kernel_set_vring(struct virtio_user_dev *dev, uint64_t req, struct vhost_v
{
int ret, fd;
uint32_t index = state->index;
+ struct vhost_kernel_data *data = dev->backend_data;
/* Convert from queue index to queue-pair & offset */
- fd = dev->vhostfds[state->index / 2];
+ fd = data->vhostfds[state->index / 2];
state->index %= 2;
ret = vhost_kernel_ioctl(fd, req, state);
@@ -265,9 +277,10 @@ vhost_kernel_set_vring_file(struct virtio_user_dev *dev, uint64_t req,
{
int ret, fd;
uint32_t index = file->index;
+ struct vhost_kernel_data *data = dev->backend_data;
/* Convert from queue index to queue-pair & offset */
- fd = dev->vhostfds[file->index / 2];
+ fd = data->vhostfds[file->index / 2];
file->index %= 2;
ret = vhost_kernel_ioctl(fd, req, file);
@@ -299,9 +312,10 @@ vhost_kernel_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr
{
int ret, fd;
uint32_t index = addr->index;
+ struct vhost_kernel_data *data = dev->backend_data;
/* Convert from queue index to queue-pair & offset */
- fd = dev->vhostfds[addr->index / 2];
+ fd = data->vhostfds[addr->index / 2];
addr->index %= 2;
ret = vhost_kernel_ioctl(fd, VHOST_SET_VRING_ADDR, addr);
@@ -339,19 +353,37 @@ static int
vhost_kernel_setup(struct virtio_user_dev *dev)
{
int vhostfd;
- uint32_t i;
+ uint32_t q, i;
+ struct vhost_kernel_data *data;
+
+ data = malloc(sizeof(*data));
+ if (!data) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data", dev->path);
+ return -1;
+ }
+
+ data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
+ data->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
+ if (!data->vhostfds || !data->tapfds) {
+ PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
+ return -1;
+ }
+
+ for (q = 0; q < dev->max_queue_pairs; ++q) {
+ data->vhostfds[q] = -1;
+ data->tapfds[q] = -1;
+ }
get_vhost_kernel_max_regions();
for (i = 0; i < dev->max_queue_pairs; ++i) {
vhostfd = open(dev->path, O_RDWR);
if (vhostfd < 0) {
- PMD_DRV_LOG(ERR, "fail to open %s, %s",
- dev->path, strerror(errno));
+ PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path, strerror(errno));
return -1;
}
- dev->vhostfds[i] = vhostfd;
+ data->vhostfds[i] = vhostfd;
}
return 0;
@@ -360,6 +392,19 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
static int
vhost_kernel_destroy(struct virtio_user_dev *dev)
{
+ struct vhost_kernel_data *data = dev->backend_data;
+ uint32_t i;
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ close(data->vhostfds[i]);
+ if (data->tapfds[i] >= 0)
+ close(data->tapfds[i]);
+ }
+
+ free(data->vhostfds);
+ free(data->tapfds);
+ free(data);
+
return 0;
}
@@ -395,14 +440,15 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
int vhostfd;
int tapfd;
int req_mq = (dev->max_queue_pairs > 1);
+ struct vhost_kernel_data *data = dev->backend_data;
- vhostfd = dev->vhostfds[pair_idx];
+ vhostfd = data->vhostfds[pair_idx];
if (dev->qp_enabled[pair_idx] == enable)
return 0;
if (!enable) {
- tapfd = dev->tapfds[pair_idx];
+ tapfd = data->tapfds[pair_idx];
if (vhost_kernel_set_backend(vhostfd, -1) < 0) {
PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
return -1;
@@ -415,8 +461,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
return 0;
}
- if (dev->tapfds[pair_idx] >= 0) {
- tapfd = dev->tapfds[pair_idx];
+ if (data->tapfds[pair_idx] >= 0) {
+ tapfd = data->tapfds[pair_idx];
if (vhost_kernel_tap_set_offload(tapfd, dev->features) == -1)
return -1;
if (req_mq && vhost_kernel_tap_set_queue(tapfd, true) < 0) {
@@ -439,7 +485,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
return -1;
}
- dev->tapfds[pair_idx] = tapfd;
+ data->tapfds[pair_idx] = tapfd;
set_backend:
if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 93a9ce2cd2..154aecc209 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -397,11 +397,6 @@ virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
static int
virtio_user_dev_setup(struct virtio_user_dev *dev)
{
- uint32_t q;
-
- dev->vhostfds = NULL;
- dev->tapfds = NULL;
-
if (dev->is_server) {
if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER) {
PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
@@ -409,34 +404,21 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
}
}
- if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
+ switch (dev->backend_type) {
+ case VIRTIO_USER_BACKEND_VHOST_USER:
dev->ops = &virtio_ops_user;
- } else if (dev->backend_type ==
- VIRTIO_USER_BACKEND_VHOST_KERNEL) {
+ break;
+ case VIRTIO_USER_BACKEND_VHOST_KERNEL:
dev->ops = &virtio_ops_kernel;
-
- dev->vhostfds = malloc(dev->max_queue_pairs *
- sizeof(int));
- dev->tapfds = malloc(dev->max_queue_pairs *
- sizeof(int));
- if (!dev->vhostfds || !dev->tapfds) {
- PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
- return -1;
- }
-
- for (q = 0; q < dev->max_queue_pairs; ++q) {
- dev->vhostfds[q] = -1;
- dev->tapfds[q] = -1;
- }
- } else if (dev->backend_type ==
- VIRTIO_USER_BACKEND_VHOST_VDPA) {
+ break;
+ case VIRTIO_USER_BACKEND_VHOST_VDPA:
dev->ops = &virtio_ops_vdpa;
- } else {
+ break;
+ default:
PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
return -1;
}
-
if (dev->ops->setup(dev) < 0) {
PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
return -1;
@@ -592,15 +574,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
close(dev->callfds[i]);
close(dev->kickfds[i]);
}
- if (dev->vhostfds) {
- for (i = 0; i < dev->max_queue_pairs; ++i) {
- close(dev->vhostfds[i]);
- if (dev->tapfds[i] >= 0)
- close(dev->tapfds[i]);
- }
- free(dev->vhostfds);
- free(dev->tapfds);
- }
free(dev->ifname);
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index a429dcc57c..5a2c9d38dd 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -31,11 +31,6 @@ struct virtio_user_dev {
/* for vhost_vdpa backend */
int vhostfd;
- /* for vhost_kernel backend */
- char *ifname;
- int *vhostfds;
- int *tapfds;
-
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES];
int kickfds[VIRTIO_MAX_VIRTQUEUES];
@@ -55,6 +50,8 @@ struct virtio_user_dev {
uint16_t port_id;
uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
char path[PATH_MAX];
+ char *ifname;
+
union {
struct vring vrings[VIRTIO_MAX_VIRTQUEUES];
struct vring_packed packed_vrings[VIRTIO_MAX_VIRTQUEUES];
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data " Maxime Coquelin
@ 2021-01-07 6:42 ` Xia, Chenbo
2021-01-11 8:02 ` Xia, Chenbo
1 sibling, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-07 6:42 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
>
> As done earlier for Vhost-user, this patch moves the
> Vhost-Kernel specific data to its backend file.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++----
> .../net/virtio/virtio_user/virtio_user_dev.c | 43 ++--------
> .../net/virtio/virtio_user/virtio_user_dev.h | 7 +-
> 3 files changed, 72 insertions(+), 56 deletions(-)
>
<snip>
> @@ -339,19 +353,37 @@ static int
> vhost_kernel_setup(struct virtio_user_dev *dev)
> {
> int vhostfd;
> - uint32_t i;
> + uint32_t q, i;
> + struct vhost_kernel_data *data;
> +
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data",
> dev->path);
> + return -1;
> + }
> +
> + data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
> + data->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
> + if (!data->vhostfds || !data->tapfds) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
> + return -1;
> + }
> +
> + for (q = 0; q < dev->max_queue_pairs; ++q) {
> + data->vhostfds[q] = -1;
> + data->tapfds[q] = -1;
> + }
>
> get_vhost_kernel_max_regions();
>
> for (i = 0; i < dev->max_queue_pairs; ++i) {
> vhostfd = open(dev->path, O_RDWR);
> if (vhostfd < 0) {
> - PMD_DRV_LOG(ERR, "fail to open %s, %s",
> - dev->path, strerror(errno));
> + PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path,
> strerror(errno));
> return -1;
> }
In this function, should we clean-up malloced resources when error happens?
Thanks,
Chenbo
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data " Maxime Coquelin
2021-01-07 6:42 ` Xia, Chenbo
@ 2021-01-11 8:02 ` Xia, Chenbo
2021-01-15 11:54 ` Maxime Coquelin
2021-01-18 20:36 ` Maxime Coquelin
1 sibling, 2 replies; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-11 8:02 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
>
> As done earlier for Vhost-user, this patch moves the
> Vhost-Kernel specific data to its backend file.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++----
> .../net/virtio/virtio_user/virtio_user_dev.c | 43 ++--------
> .../net/virtio/virtio_user/virtio_user_dev.h | 7 +-
> 3 files changed, 72 insertions(+), 56 deletions(-)
<snip>
> vhost_kernel_setup(struct virtio_user_dev *dev)
> {
> int vhostfd;
> - uint32_t i;
> + uint32_t q, i;
> + struct vhost_kernel_data *data;
> +
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data",
> dev->path);
> + return -1;
> + }
> +
> + data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
> + data->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
> + if (!data->vhostfds || !data->tapfds) {
> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
> + return -1;
> + }
> +
> + for (q = 0; q < dev->max_queue_pairs; ++q) {
> + data->vhostfds[q] = -1;
> + data->tapfds[q] = -1;
> + }
Forget to do 'dev->backend_data = data'? 😊
Besides,
When I do test 'virtio_user for exceptional path', iperf just cannot normally
send/recv pkts. Testpmd have some pkts (<10), guess tcp connection cannot be
established. I have not tried to locate the root cause. Just thinking it's
better to tell you first.
Thanks!
Chenbo
>
> get_vhost_kernel_max_regions();
>
> for (i = 0; i < dev->max_queue_pairs; ++i) {
> vhostfd = open(dev->path, O_RDWR);
> if (vhostfd < 0) {
> - PMD_DRV_LOG(ERR, "fail to open %s, %s",
> - dev->path, strerror(errno));
> + PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path,
> strerror(errno));
> return -1;
> }
>
> - dev->vhostfds[i] = vhostfd;
> + data->vhostfds[i] = vhostfd;
> }
>
> return 0;
> @@ -360,6 +392,19 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
> static int
> vhost_kernel_destroy(struct virtio_user_dev *dev)
> {
> + struct vhost_kernel_data *data = dev->backend_data;
> + uint32_t i;
> +
> + for (i = 0; i < dev->max_queue_pairs; ++i) {
> + close(data->vhostfds[i]);
> + if (data->tapfds[i] >= 0)
> + close(data->tapfds[i]);
> + }
> +
> + free(data->vhostfds);
> + free(data->tapfds);
> + free(data);
> +
> return 0;
> }
>
> @@ -395,14 +440,15 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev
> *dev,
> int vhostfd;
> int tapfd;
> int req_mq = (dev->max_queue_pairs > 1);
> + struct vhost_kernel_data *data = dev->backend_data;
>
> - vhostfd = dev->vhostfds[pair_idx];
> + vhostfd = data->vhostfds[pair_idx];
>
> if (dev->qp_enabled[pair_idx] == enable)
> return 0;
>
> if (!enable) {
> - tapfd = dev->tapfds[pair_idx];
> + tapfd = data->tapfds[pair_idx];
> if (vhost_kernel_set_backend(vhostfd, -1) < 0) {
> PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
> return -1;
> @@ -415,8 +461,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> return 0;
> }
>
> - if (dev->tapfds[pair_idx] >= 0) {
> - tapfd = dev->tapfds[pair_idx];
> + if (data->tapfds[pair_idx] >= 0) {
> + tapfd = data->tapfds[pair_idx];
> if (vhost_kernel_tap_set_offload(tapfd, dev->features) == -1)
> return -1;
> if (req_mq && vhost_kernel_tap_set_queue(tapfd, true) < 0) {
> @@ -439,7 +485,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> return -1;
> }
>
> - dev->tapfds[pair_idx] = tapfd;
> + data->tapfds[pair_idx] = tapfd;
>
> set_backend:
> if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 93a9ce2cd2..154aecc209 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -397,11 +397,6 @@ virtio_user_mem_event_cb(enum rte_mem_event type
> __rte_unused,
> static int
> virtio_user_dev_setup(struct virtio_user_dev *dev)
> {
> - uint32_t q;
> -
> - dev->vhostfds = NULL;
> - dev->tapfds = NULL;
> -
> if (dev->is_server) {
> if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER) {
> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
> @@ -409,34 +404,21 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> }
> }
>
> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
> + switch (dev->backend_type) {
> + case VIRTIO_USER_BACKEND_VHOST_USER:
> dev->ops = &virtio_ops_user;
> - } else if (dev->backend_type ==
> - VIRTIO_USER_BACKEND_VHOST_KERNEL) {
> + break;
> + case VIRTIO_USER_BACKEND_VHOST_KERNEL:
> dev->ops = &virtio_ops_kernel;
> -
> - dev->vhostfds = malloc(dev->max_queue_pairs *
> - sizeof(int));
> - dev->tapfds = malloc(dev->max_queue_pairs *
> - sizeof(int));
> - if (!dev->vhostfds || !dev->tapfds) {
> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
> - return -1;
> - }
> -
> - for (q = 0; q < dev->max_queue_pairs; ++q) {
> - dev->vhostfds[q] = -1;
> - dev->tapfds[q] = -1;
> - }
> - } else if (dev->backend_type ==
> - VIRTIO_USER_BACKEND_VHOST_VDPA) {
> + break;
> + case VIRTIO_USER_BACKEND_VHOST_VDPA:
> dev->ops = &virtio_ops_vdpa;
> - } else {
> + break;
> + default:
> PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
> return -1;
> }
>
> -
> if (dev->ops->setup(dev) < 0) {
> PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
> return -1;
> @@ -592,15 +574,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
> close(dev->callfds[i]);
> close(dev->kickfds[i]);
> }
> - if (dev->vhostfds) {
> - for (i = 0; i < dev->max_queue_pairs; ++i) {
> - close(dev->vhostfds[i]);
> - if (dev->tapfds[i] >= 0)
> - close(dev->tapfds[i]);
> - }
> - free(dev->vhostfds);
> - free(dev->tapfds);
> - }
>
> free(dev->ifname);
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index a429dcc57c..5a2c9d38dd 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -31,11 +31,6 @@ struct virtio_user_dev {
> /* for vhost_vdpa backend */
> int vhostfd;
>
> - /* for vhost_kernel backend */
> - char *ifname;
> - int *vhostfds;
> - int *tapfds;
> -
> /* for both vhost_user and vhost_kernel */
> int callfds[VIRTIO_MAX_VIRTQUEUES];
> int kickfds[VIRTIO_MAX_VIRTQUEUES];
> @@ -55,6 +50,8 @@ struct virtio_user_dev {
> uint16_t port_id;
> uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> char path[PATH_MAX];
> + char *ifname;
> +
> union {
> struct vring vrings[VIRTIO_MAX_VIRTQUEUES];
> struct vring_packed packed_vrings[VIRTIO_MAX_VIRTQUEUES];
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
2021-01-11 8:02 ` Xia, Chenbo
@ 2021-01-15 11:54 ` Maxime Coquelin
2021-01-18 20:36 ` Maxime Coquelin
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 11:54 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/11/21 9:02 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
>>
>> As done earlier for Vhost-user, this patch moves the
>> Vhost-Kernel specific data to its backend file.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++----
>> .../net/virtio/virtio_user/virtio_user_dev.c | 43 ++--------
>> .../net/virtio/virtio_user/virtio_user_dev.h | 7 +-
>> 3 files changed, 72 insertions(+), 56 deletions(-)
>
> <snip>
>
>> vhost_kernel_setup(struct virtio_user_dev *dev)
>> {
>> int vhostfd;
>> - uint32_t i;
>> + uint32_t q, i;
>> + struct vhost_kernel_data *data;
>> +
>> + data = malloc(sizeof(*data));
>> + if (!data) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data",
>> dev->path);
>> + return -1;
>> + }
>> +
>> + data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
>> + data->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
>> + if (!data->vhostfds || !data->tapfds) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
>> + return -1;
>> + }
>> +
>> + for (q = 0; q < dev->max_queue_pairs; ++q) {
>> + data->vhostfds[q] = -1;
>> + data->tapfds[q] = -1;
>> + }
>
> Forget to do 'dev->backend_data = data'? 😊
Indeed! As I mentionned in the cover letter, I didn't test last 5
patches... and we can see it confirmed right here :)
I have reworked the fucntion to handle error properly (free allocated
resources, close opened FDs...)
>
> Besides,
>
> When I do test 'virtio_user for exceptional path', iperf just cannot normally
> send/recv pkts. Testpmd have some pkts (<10), guess tcp connection cannot be
> established. I have not tried to locate the root cause. Just thinking it's
> better to tell you first.
Thanks for the heads-up, I will look into it before posting v2.
Maxime
> Thanks!
> Chenbo
>
>>
>> get_vhost_kernel_max_regions();
>>
>> for (i = 0; i < dev->max_queue_pairs; ++i) {
>> vhostfd = open(dev->path, O_RDWR);
>> if (vhostfd < 0) {
>> - PMD_DRV_LOG(ERR, "fail to open %s, %s",
>> - dev->path, strerror(errno));
>> + PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path,
>> strerror(errno));
>> return -1;
>> }
>>
>> - dev->vhostfds[i] = vhostfd;
>> + data->vhostfds[i] = vhostfd;
>> }
>>
>> return 0;
>> @@ -360,6 +392,19 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
>> static int
>> vhost_kernel_destroy(struct virtio_user_dev *dev)
>> {
>> + struct vhost_kernel_data *data = dev->backend_data;
>> + uint32_t i;
>> +
>> + for (i = 0; i < dev->max_queue_pairs; ++i) {
>> + close(data->vhostfds[i]);
>> + if (data->tapfds[i] >= 0)
>> + close(data->tapfds[i]);
>> + }
>> +
>> + free(data->vhostfds);
>> + free(data->tapfds);
>> + free(data);
>> +
>> return 0;
>> }
>>
>> @@ -395,14 +440,15 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev
>> *dev,
>> int vhostfd;
>> int tapfd;
>> int req_mq = (dev->max_queue_pairs > 1);
>> + struct vhost_kernel_data *data = dev->backend_data;
>>
>> - vhostfd = dev->vhostfds[pair_idx];
>> + vhostfd = data->vhostfds[pair_idx];
>>
>> if (dev->qp_enabled[pair_idx] == enable)
>> return 0;
>>
>> if (!enable) {
>> - tapfd = dev->tapfds[pair_idx];
>> + tapfd = data->tapfds[pair_idx];
>> if (vhost_kernel_set_backend(vhostfd, -1) < 0) {
>> PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
>> return -1;
>> @@ -415,8 +461,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
>> return 0;
>> }
>>
>> - if (dev->tapfds[pair_idx] >= 0) {
>> - tapfd = dev->tapfds[pair_idx];
>> + if (data->tapfds[pair_idx] >= 0) {
>> + tapfd = data->tapfds[pair_idx];
>> if (vhost_kernel_tap_set_offload(tapfd, dev->features) == -1)
>> return -1;
>> if (req_mq && vhost_kernel_tap_set_queue(tapfd, true) < 0) {
>> @@ -439,7 +485,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
>> return -1;
>> }
>>
>> - dev->tapfds[pair_idx] = tapfd;
>> + data->tapfds[pair_idx] = tapfd;
>>
>> set_backend:
>> if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index 93a9ce2cd2..154aecc209 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -397,11 +397,6 @@ virtio_user_mem_event_cb(enum rte_mem_event type
>> __rte_unused,
>> static int
>> virtio_user_dev_setup(struct virtio_user_dev *dev)
>> {
>> - uint32_t q;
>> -
>> - dev->vhostfds = NULL;
>> - dev->tapfds = NULL;
>> -
>> if (dev->is_server) {
>> if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER) {
>> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
>> @@ -409,34 +404,21 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
>> }
>> }
>>
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
>> + switch (dev->backend_type) {
>> + case VIRTIO_USER_BACKEND_VHOST_USER:
>> dev->ops = &virtio_ops_user;
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_KERNEL) {
>> + break;
>> + case VIRTIO_USER_BACKEND_VHOST_KERNEL:
>> dev->ops = &virtio_ops_kernel;
>> -
>> - dev->vhostfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - dev->tapfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - if (!dev->vhostfds || !dev->tapfds) {
>> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
>> - return -1;
>> - }
>> -
>> - for (q = 0; q < dev->max_queue_pairs; ++q) {
>> - dev->vhostfds[q] = -1;
>> - dev->tapfds[q] = -1;
>> - }
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_VDPA) {
>> + break;
>> + case VIRTIO_USER_BACKEND_VHOST_VDPA:
>> dev->ops = &virtio_ops_vdpa;
>> - } else {
>> + break;
>> + default:
>> PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
>> return -1;
>> }
>>
>> -
>> if (dev->ops->setup(dev) < 0) {
>> PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
>> return -1;
>> @@ -592,15 +574,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
>> close(dev->callfds[i]);
>> close(dev->kickfds[i]);
>> }
>> - if (dev->vhostfds) {
>> - for (i = 0; i < dev->max_queue_pairs; ++i) {
>> - close(dev->vhostfds[i]);
>> - if (dev->tapfds[i] >= 0)
>> - close(dev->tapfds[i]);
>> - }
>> - free(dev->vhostfds);
>> - free(dev->tapfds);
>> - }
>>
>> free(dev->ifname);
>>
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> index a429dcc57c..5a2c9d38dd 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> @@ -31,11 +31,6 @@ struct virtio_user_dev {
>> /* for vhost_vdpa backend */
>> int vhostfd;
>>
>> - /* for vhost_kernel backend */
>> - char *ifname;
>> - int *vhostfds;
>> - int *tapfds;
>> -
>> /* for both vhost_user and vhost_kernel */
>> int callfds[VIRTIO_MAX_VIRTQUEUES];
>> int kickfds[VIRTIO_MAX_VIRTQUEUES];
>> @@ -55,6 +50,8 @@ struct virtio_user_dev {
>> uint16_t port_id;
>> uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
>> char path[PATH_MAX];
>> + char *ifname;
>> +
>> union {
>> struct vring vrings[VIRTIO_MAX_VIRTQUEUES];
>> struct vring_packed packed_vrings[VIRTIO_MAX_VIRTQUEUES];
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
2021-01-11 8:02 ` Xia, Chenbo
2021-01-15 11:54 ` Maxime Coquelin
@ 2021-01-18 20:36 ` Maxime Coquelin
1 sibling, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-18 20:36 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/11/21 9:02 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend
>>
>> As done earlier for Vhost-user, this patch moves the
>> Vhost-Kernel specific data to its backend file.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++----
>> .../net/virtio/virtio_user/virtio_user_dev.c | 43 ++--------
>> .../net/virtio/virtio_user/virtio_user_dev.h | 7 +-
>> 3 files changed, 72 insertions(+), 56 deletions(-)
>
> <snip>
>
>> vhost_kernel_setup(struct virtio_user_dev *dev)
>> {
>> int vhostfd;
>> - uint32_t i;
>> + uint32_t q, i;
>> + struct vhost_kernel_data *data;
>> +
>> + data = malloc(sizeof(*data));
>> + if (!data) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data",
>> dev->path);
>> + return -1;
>> + }
>> +
>> + data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
>> + data->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
>> + if (!data->vhostfds || !data->tapfds) {
>> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
>> + return -1;
>> + }
>> +
>> + for (q = 0; q < dev->max_queue_pairs; ++q) {
>> + data->vhostfds[q] = -1;
>> + data->tapfds[q] = -1;
>> + }
>
> Forget to do 'dev->backend_data = data'? 😊
>
> Besides,
>
> When I do test 'virtio_user for exceptional path', iperf just cannot normally
> send/recv pkts. Testpmd have some pkts (<10), guess tcp connection cannot be
> established. I have not tried to locate the root cause. Just thinking it's
> better to tell you first.
Thanks for reporting.
I did a git bisect and could find the issue:
static int
vhost_kernel_set_vring_kick(struct virtio_user_dev *dev, struct
vhost_vring_file *file)
{
return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_CALL, file);
}
static int
vhost_kernel_set_vring_call(struct virtio_user_dev *dev, struct
vhost_vring_file *file)
{
return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file);
}
The kick and call request were inverted!
Once fixed, I can txonly without getting stuck once queue is full.
I will run more Vhost-kernel tests, but I think that was the issue you
found.
Maxime
> Thanks!
> Chenbo
>
>>
>> get_vhost_kernel_max_regions();
>>
>> for (i = 0; i < dev->max_queue_pairs; ++i) {
>> vhostfd = open(dev->path, O_RDWR);
>> if (vhostfd < 0) {
>> - PMD_DRV_LOG(ERR, "fail to open %s, %s",
>> - dev->path, strerror(errno));
>> + PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path,
>> strerror(errno));
>> return -1;
>> }
>>
>> - dev->vhostfds[i] = vhostfd;
>> + data->vhostfds[i] = vhostfd;
>> }
>>
>> return 0;
>> @@ -360,6 +392,19 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
>> static int
>> vhost_kernel_destroy(struct virtio_user_dev *dev)
>> {
>> + struct vhost_kernel_data *data = dev->backend_data;
>> + uint32_t i;
>> +
>> + for (i = 0; i < dev->max_queue_pairs; ++i) {
>> + close(data->vhostfds[i]);
>> + if (data->tapfds[i] >= 0)
>> + close(data->tapfds[i]);
>> + }
>> +
>> + free(data->vhostfds);
>> + free(data->tapfds);
>> + free(data);
>> +
>> return 0;
>> }
>>
>> @@ -395,14 +440,15 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev
>> *dev,
>> int vhostfd;
>> int tapfd;
>> int req_mq = (dev->max_queue_pairs > 1);
>> + struct vhost_kernel_data *data = dev->backend_data;
>>
>> - vhostfd = dev->vhostfds[pair_idx];
>> + vhostfd = data->vhostfds[pair_idx];
>>
>> if (dev->qp_enabled[pair_idx] == enable)
>> return 0;
>>
>> if (!enable) {
>> - tapfd = dev->tapfds[pair_idx];
>> + tapfd = data->tapfds[pair_idx];
>> if (vhost_kernel_set_backend(vhostfd, -1) < 0) {
>> PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
>> return -1;
>> @@ -415,8 +461,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
>> return 0;
>> }
>>
>> - if (dev->tapfds[pair_idx] >= 0) {
>> - tapfd = dev->tapfds[pair_idx];
>> + if (data->tapfds[pair_idx] >= 0) {
>> + tapfd = data->tapfds[pair_idx];
>> if (vhost_kernel_tap_set_offload(tapfd, dev->features) == -1)
>> return -1;
>> if (req_mq && vhost_kernel_tap_set_queue(tapfd, true) < 0) {
>> @@ -439,7 +485,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
>> return -1;
>> }
>>
>> - dev->tapfds[pair_idx] = tapfd;
>> + data->tapfds[pair_idx] = tapfd;
>>
>> set_backend:
>> if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> index 93a9ce2cd2..154aecc209 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
>> @@ -397,11 +397,6 @@ virtio_user_mem_event_cb(enum rte_mem_event type
>> __rte_unused,
>> static int
>> virtio_user_dev_setup(struct virtio_user_dev *dev)
>> {
>> - uint32_t q;
>> -
>> - dev->vhostfds = NULL;
>> - dev->tapfds = NULL;
>> -
>> if (dev->is_server) {
>> if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER) {
>> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!");
>> @@ -409,34 +404,21 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
>> }
>> }
>>
>> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) {
>> + switch (dev->backend_type) {
>> + case VIRTIO_USER_BACKEND_VHOST_USER:
>> dev->ops = &virtio_ops_user;
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_KERNEL) {
>> + break;
>> + case VIRTIO_USER_BACKEND_VHOST_KERNEL:
>> dev->ops = &virtio_ops_kernel;
>> -
>> - dev->vhostfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - dev->tapfds = malloc(dev->max_queue_pairs *
>> - sizeof(int));
>> - if (!dev->vhostfds || !dev->tapfds) {
>> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path);
>> - return -1;
>> - }
>> -
>> - for (q = 0; q < dev->max_queue_pairs; ++q) {
>> - dev->vhostfds[q] = -1;
>> - dev->tapfds[q] = -1;
>> - }
>> - } else if (dev->backend_type ==
>> - VIRTIO_USER_BACKEND_VHOST_VDPA) {
>> + break;
>> + case VIRTIO_USER_BACKEND_VHOST_VDPA:
>> dev->ops = &virtio_ops_vdpa;
>> - } else {
>> + break;
>> + default:
>> PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path);
>> return -1;
>> }
>>
>> -
>> if (dev->ops->setup(dev) < 0) {
>> PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path);
>> return -1;
>> @@ -592,15 +574,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
>> close(dev->callfds[i]);
>> close(dev->kickfds[i]);
>> }
>> - if (dev->vhostfds) {
>> - for (i = 0; i < dev->max_queue_pairs; ++i) {
>> - close(dev->vhostfds[i]);
>> - if (dev->tapfds[i] >= 0)
>> - close(dev->tapfds[i]);
>> - }
>> - free(dev->vhostfds);
>> - free(dev->tapfds);
>> - }
>>
>> free(dev->ifname);
>>
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> index a429dcc57c..5a2c9d38dd 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> @@ -31,11 +31,6 @@ struct virtio_user_dev {
>> /* for vhost_vdpa backend */
>> int vhostfd;
>>
>> - /* for vhost_kernel backend */
>> - char *ifname;
>> - int *vhostfds;
>> - int *tapfds;
>> -
>> /* for both vhost_user and vhost_kernel */
>> int callfds[VIRTIO_MAX_VIRTQUEUES];
>> int kickfds[VIRTIO_MAX_VIRTQUEUES];
>> @@ -55,6 +50,8 @@ struct virtio_user_dev {
>> uint16_t port_id;
>> uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
>> char path[PATH_MAX];
>> + char *ifname;
>> +
>> union {
>> struct vring vrings[VIRTIO_MAX_VIRTQUEUES];
>> struct vring_packed packed_vrings[VIRTIO_MAX_VIRTQUEUES];
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (38 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data " Maxime Coquelin
@ 2020-12-20 21:14 ` Maxime Coquelin
2020-12-22 15:20 ` Maxime Coquelin
` (2 more replies)
2020-12-21 10:58 ` [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
40 siblings, 3 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-20 21:14 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand; +Cc: Maxime Coquelin
As done earlier for Vhost-user and Vhost-kernel, this
patch moves the Vhost-vDPA specific data to its backend
file.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/vhost_vdpa.c | 77 ++++++++++++++-----
.../net/virtio/virtio_user/virtio_user_dev.h | 3 -
2 files changed, 58 insertions(+), 22 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_vdpa.c b/drivers/net/virtio/virtio_user/vhost_vdpa.c
index 0f422ae84a..c9738b462a 100644
--- a/drivers/net/virtio/virtio_user/vhost_vdpa.c
+++ b/drivers/net/virtio/virtio_user/vhost_vdpa.c
@@ -13,6 +13,10 @@
#include "vhost.h"
#include "virtio_user_dev.h"
+struct vhost_vdpa_data {
+ int vhostfd;
+};
+
/* vhost kernel & vdpa ioctls */
#define VHOST_VIRTIO 0xAF
#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
@@ -80,15 +84,18 @@ vhost_vdpa_ioctl(int fd, uint64_t request, void *arg)
static int
vhost_vdpa_set_owner(struct virtio_user_dev *dev)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_OWNER, NULL);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_OWNER, NULL);
}
static int
vhost_vdpa_get_features(struct virtio_user_dev *dev, uint64_t *features)
{
+ struct vhost_vdpa_data *data = dev->backend_data;
int ret;
- ret = vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_FEATURES, features);
+ ret = vhost_vdpa_ioctl(data->vhostfd, VHOST_GET_FEATURES, features);
if (ret) {
PMD_DRV_LOG(ERR, "Failed to get features");
return -1;
@@ -103,16 +110,19 @@ vhost_vdpa_get_features(struct virtio_user_dev *dev, uint64_t *features)
static int
vhost_vdpa_set_features(struct virtio_user_dev *dev, uint64_t features)
{
+ struct vhost_vdpa_data *data = dev->backend_data;
+
/* WORKAROUND */
features |= 1ULL << VIRTIO_F_IOMMU_PLATFORM;
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_FEATURES, &features);
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_FEATURES, &features);
}
static int
vhost_vdpa_dma_map(struct virtio_user_dev *dev, void *addr,
uint64_t iova, size_t len)
{
+ struct vhost_vdpa_data *data = dev->backend_data;
struct vhost_msg msg = {};
msg.type = VHOST_IOTLB_MSG_V2;
@@ -122,7 +132,7 @@ vhost_vdpa_dma_map(struct virtio_user_dev *dev, void *addr,
msg.iotlb.size = len;
msg.iotlb.perm = VHOST_ACCESS_RW;
- if (write(dev->vhostfd, &msg, sizeof(msg)) != sizeof(msg)) {
+ if (write(data->vhostfd, &msg, sizeof(msg)) != sizeof(msg)) {
PMD_DRV_LOG(ERR, "Failed to send IOTLB update (%s)",
strerror(errno));
return -1;
@@ -135,6 +145,7 @@ static int
vhost_vdpa_dma_unmap(struct virtio_user_dev *dev, __rte_unused void *addr,
uint64_t iova, size_t len)
{
+ struct vhost_vdpa_data *data = dev->backend_data;
struct vhost_msg msg = {};
msg.type = VHOST_IOTLB_MSG_V2;
@@ -142,7 +153,7 @@ vhost_vdpa_dma_unmap(struct virtio_user_dev *dev, __rte_unused void *addr,
msg.iotlb.iova = iova;
msg.iotlb.size = len;
- if (write(dev->vhostfd, &msg, sizeof(msg)) != sizeof(msg)) {
+ if (write(data->vhostfd, &msg, sizeof(msg)) != sizeof(msg)) {
PMD_DRV_LOG(ERR, "Failed to send IOTLB invalidate (%s)",
strerror(errno));
return -1;
@@ -208,55 +219,73 @@ vhost_vdpa_set_memory_table(struct virtio_user_dev *dev)
static int
vhost_vdpa_set_vring_enable(struct virtio_user_dev *dev, struct vhost_vring_state *state)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_VRING_ENABLE, state);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_VDPA_SET_VRING_ENABLE, state);
}
static int
vhost_vdpa_set_vring_num(struct virtio_user_dev *dev, struct vhost_vring_state *state)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_NUM, state);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_VRING_NUM, state);
}
static int
vhost_vdpa_set_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_BASE, state);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_VRING_BASE, state);
}
static int
vhost_vdpa_get_vring_base(struct virtio_user_dev *dev, struct vhost_vring_state *state)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_GET_VRING_BASE, state);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_GET_VRING_BASE, state);
}
static int
vhost_vdpa_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file *file)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_CALL, file);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_VRING_CALL, file);
}
static int
vhost_vdpa_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *file)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_KICK, file);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_VRING_KICK, file);
}
static int
vhost_vdpa_set_vring_addr(struct virtio_user_dev *dev, struct vhost_vring_addr *addr)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_SET_VRING_ADDR, addr);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_SET_VRING_ADDR, addr);
}
static int
vhost_vdpa_get_status(struct virtio_user_dev *dev, uint8_t *status)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_GET_STATUS, status);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_VDPA_GET_STATUS, status);
}
static int
vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
{
- return vhost_vdpa_ioctl(dev->vhostfd, VHOST_VDPA_SET_STATUS, &status);
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ return vhost_vdpa_ioctl(data->vhostfd, VHOST_VDPA_SET_STATUS, &status);
}
/**
@@ -269,16 +298,23 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
static int
vhost_vdpa_setup(struct virtio_user_dev *dev)
{
+ struct vhost_vdpa_data *data;
uint32_t did = (uint32_t)-1;
- dev->vhostfd = open(dev->path, O_RDWR);
- if (dev->vhostfd < 0) {
+ data = malloc(sizeof(*data));
+ if (!data) {
+ PMD_DRV_LOG(ERR, "(%s) Faidle to allocate backend data", dev->path);
+ return -1;
+ }
+
+ data->vhostfd = open(dev->path, O_RDWR);
+ if (data->vhostfd < 0) {
PMD_DRV_LOG(ERR, "Failed to open %s: %s\n",
dev->path, strerror(errno));
return -1;
}
- if (ioctl(dev->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
+ if (ioctl(data->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
did != VIRTIO_ID_NETWORK) {
PMD_DRV_LOG(ERR, "Invalid vdpa device ID: %u\n", did);
return -1;
@@ -288,9 +324,12 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
}
static int
-vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
+vhost_vdpa_destroy(struct virtio_user_dev *dev )
{
- return;
+ struct vhost_vdpa_data *data = dev->backend_data;
+
+ close(data->vhostfd);
+
return 0;
}
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 5a2c9d38dd..2e0d6504f6 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -28,9 +28,6 @@ struct virtio_user_dev {
enum virtio_user_backend_type backend_type;
bool is_server; /* server or client mode */
- /* for vhost_vdpa backend */
- int vhostfd;
-
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES];
int kickfds[VIRTIO_MAX_VIRTQUEUES];
--
2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA " Maxime Coquelin
@ 2020-12-22 15:20 ` Maxime Coquelin
2021-01-07 6:50 ` Xia, Chenbo
2021-01-11 8:05 ` Xia, Chenbo
2 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-22 15:20 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand
On 12/20/20 10:14 PM, Maxime Coquelin wrote:
> As done earlier for Vhost-user and Vhost-kernel, this
> patch moves the Vhost-vDPA specific data to its backend
> file.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 77 ++++++++++++++-----
> .../net/virtio/virtio_user/virtio_user_dev.h | 3 -
> 2 files changed, 58 insertions(+), 22 deletions(-)
>
...
> @@ -269,16 +298,23 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev, uint8_t status)
> static int
> vhost_vdpa_setup(struct virtio_user_dev *dev)
> {
> + struct vhost_vdpa_data *data;
> uint32_t did = (uint32_t)-1;
>
> - dev->vhostfd = open(dev->path, O_RDWR);
> - if (dev->vhostfd < 0) {
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_DRV_LOG(ERR, "(%s) Faidle to allocate backend data", dev->path);
> + return -1;
> + }
> +
> + data->vhostfd = open(dev->path, O_RDWR);
> + if (data->vhostfd < 0) {
> PMD_DRV_LOG(ERR, "Failed to open %s: %s\n",
> dev->path, strerror(errno));
> return -1;
> }
>
> - if (ioctl(dev->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> + if (ioctl(data->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> did != VIRTIO_ID_NETWORK) {
> PMD_DRV_LOG(ERR, "Invalid vdpa device ID: %u\n", did);
> return -1;
> @@ -288,9 +324,12 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
> }
>
> static int
> -vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
> +vhost_vdpa_destroy(struct virtio_user_dev *dev )
> {
> - return;
> + struct vhost_vdpa_data *data = dev->backend_data;
> +
> + close(data->vhostfd);
> +
Note to self: free(data); here.
> return 0;
> }
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 5a2c9d38dd..2e0d6504f6 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -28,9 +28,6 @@ struct virtio_user_dev {
> enum virtio_user_backend_type backend_type;
> bool is_server; /* server or client mode */
>
> - /* for vhost_vdpa backend */
> - int vhostfd;
> -
> /* for both vhost_user and vhost_kernel */
> int callfds[VIRTIO_MAX_VIRTQUEUES];
> int kickfds[VIRTIO_MAX_VIRTQUEUES];
>
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA " Maxime Coquelin
2020-12-22 15:20 ` Maxime Coquelin
@ 2021-01-07 6:50 ` Xia, Chenbo
2021-01-15 12:08 ` Maxime Coquelin
2021-01-11 8:05 ` Xia, Chenbo
2 siblings, 1 reply; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-07 6:50 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
>
> As done earlier for Vhost-user and Vhost-kernel, this
> patch moves the Vhost-vDPA specific data to its backend
> file.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 77 ++++++++++++++-----
> .../net/virtio/virtio_user/virtio_user_dev.h | 3 -
> 2 files changed, 58 insertions(+), 22 deletions(-)
>
<snip>
> @@ -269,16 +298,23 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev,
> uint8_t status)
> static int
> vhost_vdpa_setup(struct virtio_user_dev *dev)
> {
> + struct vhost_vdpa_data *data;
> uint32_t did = (uint32_t)-1;
>
> - dev->vhostfd = open(dev->path, O_RDWR);
> - if (dev->vhostfd < 0) {
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_DRV_LOG(ERR, "(%s) Faidle to allocate backend data", dev-
> >path);
> + return -1;
> + }
> +
> + data->vhostfd = open(dev->path, O_RDWR);
> + if (data->vhostfd < 0) {
> PMD_DRV_LOG(ERR, "Failed to open %s: %s\n",
> dev->path, strerror(errno));
> return -1;
> }
>
> - if (ioctl(dev->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> + if (ioctl(data->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> did != VIRTIO_ID_NETWORK) {
> PMD_DRV_LOG(ERR, "Invalid vdpa device ID: %u\n", did);
> return -1;
Need to clean up resources when error happens?
> @@ -288,9 +324,12 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
> }
>
> static int
> -vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
> +vhost_vdpa_destroy(struct virtio_user_dev *dev )
Should delete the space before ')'?
> {
> - return;
> + struct vhost_vdpa_data *data = dev->backend_data;
> +
> + close(data->vhostfd);
> +
> return 0;
> }
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 5a2c9d38dd..2e0d6504f6 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -28,9 +28,6 @@ struct virtio_user_dev {
> enum virtio_user_backend_type backend_type;
> bool is_server; /* server or client mode */
>
> - /* for vhost_vdpa backend */
> - int vhostfd;
> -
> /* for both vhost_user and vhost_kernel */
Is this comment no longer needed too?
Thanks!
Chenbo
> int callfds[VIRTIO_MAX_VIRTQUEUES];
> int kickfds[VIRTIO_MAX_VIRTQUEUES];
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread* Re: [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
2021-01-07 6:50 ` Xia, Chenbo
@ 2021-01-15 12:08 ` Maxime Coquelin
0 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2021-01-15 12:08 UTC (permalink / raw)
To: Xia, Chenbo, dev, olivier.matz, amorenoz, david.marchand
On 1/7/21 7:50 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, December 21, 2020 5:14 AM
>> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
>> amorenoz@redhat.com; david.marchand@redhat.com
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
>>
>> As done earlier for Vhost-user and Vhost-kernel, this
>> patch moves the Vhost-vDPA specific data to its backend
>> file.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> drivers/net/virtio/virtio_user/vhost_vdpa.c | 77 ++++++++++++++-----
>> .../net/virtio/virtio_user/virtio_user_dev.h | 3 -
>> 2 files changed, 58 insertions(+), 22 deletions(-)
>>
>
> <snip>
>
>> @@ -269,16 +298,23 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev,
>> uint8_t status)
>> static int
>> vhost_vdpa_setup(struct virtio_user_dev *dev)
>> {
>> + struct vhost_vdpa_data *data;
>> uint32_t did = (uint32_t)-1;
>>
>> - dev->vhostfd = open(dev->path, O_RDWR);
>> - if (dev->vhostfd < 0) {
>> + data = malloc(sizeof(*data));
>> + if (!data) {
>> + PMD_DRV_LOG(ERR, "(%s) Faidle to allocate backend data", dev-
>>> path);
>> + return -1;
>> + }
>> +
>> + data->vhostfd = open(dev->path, O_RDWR);
>> + if (data->vhostfd < 0) {
>> PMD_DRV_LOG(ERR, "Failed to open %s: %s\n",
>> dev->path, strerror(errno));
>> return -1;
>> }
>>
>> - if (ioctl(dev->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
>> + if (ioctl(data->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
>> did != VIRTIO_ID_NETWORK) {
>> PMD_DRV_LOG(ERR, "Invalid vdpa device ID: %u\n", did);
>> return -1;
>
> Need to clean up resources when error happens?
Yes, and also assign ddev->backend_data in case of success.
>
>> @@ -288,9 +324,12 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
>> }
>>
>> static int
>> -vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
>> +vhost_vdpa_destroy(struct virtio_user_dev *dev )
>
> Should delete the space before ')'?
Done
>> {
>> - return;
>> + struct vhost_vdpa_data *data = dev->backend_data;
>> +
>> + close(data->vhostfd);
>> +
>> return 0;
>> }
>>
>> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> index 5a2c9d38dd..2e0d6504f6 100644
>> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
>> @@ -28,9 +28,6 @@ struct virtio_user_dev {
>> enum virtio_user_backend_type backend_type;
>> bool is_server; /* server or client mode */
>>
>> - /* for vhost_vdpa backend */
>> - int vhostfd;
>> -
>> /* for both vhost_user and vhost_kernel */
>
> Is this comment no longer needed too?
Removed.
Thanks!
Maxime
> Thanks!
> Chenbo
>
>> int callfds[VIRTIO_MAX_VIRTQUEUES];
>> int kickfds[VIRTIO_MAX_VIRTQUEUES];
>> --
>> 2.29.2
>
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
2020-12-20 21:14 ` [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA " Maxime Coquelin
2020-12-22 15:20 ` Maxime Coquelin
2021-01-07 6:50 ` Xia, Chenbo
@ 2021-01-11 8:05 ` Xia, Chenbo
2 siblings, 0 replies; 149+ messages in thread
From: Xia, Chenbo @ 2021-01-11 8:05 UTC (permalink / raw)
To: Maxime Coquelin, dev, olivier.matz, amorenoz, david.marchand
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, December 21, 2020 5:14 AM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; olivier.matz@6wind.com;
> amorenoz@redhat.com; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 40/40] net/virtio: move Vhost-vDPA data to its backend
>
> As done earlier for Vhost-user and Vhost-kernel, this
> patch moves the Vhost-vDPA specific data to its backend
> file.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 77 ++++++++++++++-----
> .../net/virtio/virtio_user/virtio_user_dev.h | 3 -
> 2 files changed, 58 insertions(+), 22 deletions(-)
<snip>
> @@ -269,16 +298,23 @@ vhost_vdpa_set_status(struct virtio_user_dev *dev,
> uint8_t status)
> static int
> vhost_vdpa_setup(struct virtio_user_dev *dev)
> {
> + struct vhost_vdpa_data *data;
> uint32_t did = (uint32_t)-1;
>
> - dev->vhostfd = open(dev->path, O_RDWR);
> - if (dev->vhostfd < 0) {
> + data = malloc(sizeof(*data));
> + if (!data) {
> + PMD_DRV_LOG(ERR, "(%s) Faidle to allocate backend data", dev-
> >path);
> + return -1;
> + }
> +
> + data->vhostfd = open(dev->path, O_RDWR);
> + if (data->vhostfd < 0) {
> PMD_DRV_LOG(ERR, "Failed to open %s: %s\n",
> dev->path, strerror(errno));
> return -1;
> }
>
Forget to do 'dev->backend_data = data'? 😊
Thanks!
Chenbo
> - if (ioctl(dev->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> + if (ioctl(data->vhostfd, VHOST_VDPA_GET_DEVICE_ID, &did) < 0 ||
> did != VIRTIO_ID_NETWORK) {
> PMD_DRV_LOG(ERR, "Invalid vdpa device ID: %u\n", did);
> return -1;
> @@ -288,9 +324,12 @@ vhost_vdpa_setup(struct virtio_user_dev *dev)
> }
>
> static int
> -vhost_vdpa_destroy(struct virtio_user_dev *dev __rte_unused)
> +vhost_vdpa_destroy(struct virtio_user_dev *dev )
> {
> - return;
> + struct vhost_vdpa_data *data = dev->backend_data;
> +
> + close(data->vhostfd);
> +
> return 0;
> }
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 5a2c9d38dd..2e0d6504f6 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -28,9 +28,6 @@ struct virtio_user_dev {
> enum virtio_user_backend_type backend_type;
> bool is_server; /* server or client mode */
>
> - /* for vhost_vdpa backend */
> - int vhostfd;
> -
> /* for both vhost_user and vhost_kernel */
> int callfds[VIRTIO_MAX_VIRTQUEUES];
> int kickfds[VIRTIO_MAX_VIRTQUEUES];
> --
> 2.29.2
^ permalink raw reply [flat|nested] 149+ messages in thread
* Re: [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework
2020-12-20 21:13 [dpdk-dev] [PATCH 00/40] net/virtio: Virtio PMD rework Maxime Coquelin
` (39 preceding siblings ...)
2020-12-20 21:14 ` [dpdk-dev] [PATCH 40/40] net/virtio: move Vhost-vDPA " Maxime Coquelin
@ 2020-12-21 10:58 ` Maxime Coquelin
40 siblings, 0 replies; 149+ messages in thread
From: Maxime Coquelin @ 2020-12-21 10:58 UTC (permalink / raw)
To: dev, chenbo.xia, olivier.matz, amorenoz, david.marchand
On 12/20/20 10:13 PM, Maxime Coquelin wrote:
> This series significantly rework Virtio PMD to improve
> the Virtio-user PMD and its backends integration.
>
> First part of the series (first 21 patches) removes the
> dependency of Virtio-user ethdev on Virtio PCI, by
> creating generic files, adding per-bus meta data, ...
>
> Main (if not single) functionnal change of this first
> part is to remove the hack for Virtio-user to work in
> IOVA as PA mode, this hack being very fragile. Now, the
> user has to manually pass --iova-mode=va in EAL
> parameters, otherwise vdev probe will fail. In v21.11,
> when ABI/API can be changed, I will add vdev driver
> flags so that the Virtio-user PMD can request IOVA as VA
> mode to be used.
>
> Second part of the series reworks Virtio-user internal,
> by reworking the requests handling so that vDPA and Kernel
> backends no more hack into being Vhost-user backend. It
> implies implementing new ops for all the request types.
> Also, all the backend specific actions are moved from the
> virtio_user_dev.c and virtio_user_ethdev.c to their
> backend files.
>
> Only functionnal change in this second part is making the
> Vhost-user server mode blocking at init time, as long as
> a client is not connected. The goal of this change is to
> make the Vhost-user support much more robust, as without
> blocking, the driver has to assume features that are going
> to be supported by the client, which is very fragile and
> error prone. As a side-effect, it also simplifies the
> logic nin several place of the virtio-user PMD.
>
> Plese note that I haven't tested the last 5 patches yet,
> I will conduct more testing early next week.
I forgot to add the remaining things to do in next release:
1. More testing
2. Rebase on top of Vhost-vDPA batching support
3. Rebase on top of Olivier's protocol features fix
4. (Maybe) Loosen restrictions on IOVA as VA mode, by making Vhost-
backend to use IOVA instead of directly VAs, but still warn IOVA
as VA mode is advised to ensure init won't fail.
Regards,
Maxime
^ permalink raw reply [flat|nested] 149+ messages in thread