* [patch v4 0/6] Support VMBUS channels without monitoring enabled
@ 2025-04-07 22:44 longli
2025-04-07 22:44 ` [patch v4 1/6] net/netvsc: introduce private data for storing vmbus device for secondary process longli
` (5 more replies)
0 siblings, 6 replies; 9+ messages in thread
From: longli @ 2025-04-07 22:44 UTC (permalink / raw)
To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li
From: Long Li <longli@microsoft.com>
Hyperv may expose VMBUS channels without monitoring enabled. In this case,
it programs almost all the data traffic to VF.
This patchset enabled vmbus/netvsc to use channels without monitoring
enabled.
This needs to change the APIs exposed by drivers/bus/vmbus. Because those
APIs are used only by NetVSC PMD and not feasible for use by a DPDK
application, all VMBUS APIs are set to deprecate in upcoming 25.11 release.
The notice for deprecation is accepted in the following patch:
https://patchwork.dpdk.org/project/dpdk/patch/1742242184-19600-1-git-send-email-longli@linuxonhyperv.com/
Change log:
v3: Add in the comment on the VMBUS API deprecation notice
Long Li (6):
net/netvsc: introduce private data for storing vmbus device for
secondary process
net/netvsc: introduce get_vmbus_device to get the vmbus device
bus/vmbus: store UIO fd for secondary process
bus/vmbus: support channels without monitoring enabled
bus/vmbus: add rte_vmbus_device to all functions accessing vmbus
bus/vmbus: set event for channel without monitoring support
drivers/bus/vmbus/linux/vmbus_bus.c | 9 +++--
drivers/bus/vmbus/linux/vmbus_uio.c | 6 +--
drivers/bus/vmbus/private.h | 2 +-
drivers/bus/vmbus/rte_bus_vmbus.h | 16 +++++---
drivers/bus/vmbus/vmbus_channel.c | 57 ++++++++++++++++++++--------
drivers/bus/vmbus/vmbus_common_uio.c | 9 +++--
drivers/net/netvsc/hn_ethdev.c | 44 ++++++++++++++++-----
drivers/net/netvsc/hn_nvs.c | 33 +++++++++++-----
drivers/net/netvsc/hn_nvs.h | 21 ++++++----
drivers/net/netvsc/hn_rndis.c | 11 +++---
drivers/net/netvsc/hn_rxtx.c | 16 ++++----
11 files changed, 155 insertions(+), 69 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [patch v4 1/6] net/netvsc: introduce private data for storing vmbus device for secondary process 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli @ 2025-04-07 22:44 ` longli 2025-04-07 22:45 ` [patch v4 2/6] net/netvsc: introduce get_vmbus_device to get the vmbus device longli ` (4 subsequent siblings) 5 siblings, 0 replies; 9+ messages in thread From: longli @ 2025-04-07 22:44 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> To prepare for supporting to set hyperv event from secondary process when the channel has monitoring disable, introduce a private data region for storing the vmbus device. The secondary process will get access to its vmbus device in case it needs to signal the host. Signed-off-by: Long Li <longli@microsoft.com> --- drivers/net/netvsc/hn_ethdev.c | 44 +++++++++++++++++++++++++++------- drivers/net/netvsc/hn_nvs.h | 4 ++++ 2 files changed, 39 insertions(+), 9 deletions(-) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index f848157b49..e5b052d569 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -1427,7 +1427,8 @@ static int eth_hn_probe(struct rte_vmbus_driver *drv __rte_unused, struct rte_vmbus_device *dev) { struct rte_eth_dev *eth_dev; - int ret; + struct hn_nvs_process_priv *process_priv; + int ret = 0; PMD_INIT_FUNC_TRACE(); @@ -1438,16 +1439,37 @@ static int eth_hn_probe(struct rte_vmbus_driver *drv __rte_unused, } eth_dev = eth_dev_vmbus_allocate(dev, sizeof(struct hn_data)); - if (!eth_dev) - return -ENOMEM; + if (!eth_dev) { + ret = -ENOMEM; + goto vmbus_alloc_failed; + } - ret = eth_hn_dev_init(eth_dev); - if (ret) { - eth_dev_vmbus_release(eth_dev); - rte_dev_event_monitor_stop(); - } else { - rte_eth_dev_probing_finish(eth_dev); + process_priv = rte_zmalloc_socket("netvsc_proc_priv", + sizeof(struct hn_nvs_process_priv), + RTE_CACHE_LINE_SIZE, + dev->device.numa_node); + if (!process_priv) { + ret = -ENOMEM; + goto priv_alloc_failed; } + process_priv->vmbus_dev = dev; + eth_dev->process_private = process_priv; + + ret = eth_hn_dev_init(eth_dev); + if (ret) + goto dev_init_failed; + + rte_eth_dev_probing_finish(eth_dev); + return ret; + +dev_init_failed: + rte_free(process_priv); + +priv_alloc_failed: + eth_dev_vmbus_release(eth_dev); + +vmbus_alloc_failed: + rte_dev_event_monitor_stop(); return ret; } @@ -1455,6 +1477,7 @@ static int eth_hn_probe(struct rte_vmbus_driver *drv __rte_unused, static int eth_hn_remove(struct rte_vmbus_device *dev) { struct rte_eth_dev *eth_dev; + struct hn_nvs_process_priv *process_priv; int ret; PMD_INIT_FUNC_TRACE(); @@ -1467,6 +1490,9 @@ static int eth_hn_remove(struct rte_vmbus_device *dev) if (ret) return ret; + process_priv = eth_dev->process_private; + rte_free(process_priv); + eth_dev_vmbus_release(eth_dev); rte_dev_event_monitor_stop(); return 0; diff --git a/drivers/net/netvsc/hn_nvs.h b/drivers/net/netvsc/hn_nvs.h index 2843ef7b96..3950749359 100644 --- a/drivers/net/netvsc/hn_nvs.h +++ b/drivers/net/netvsc/hn_nvs.h @@ -65,6 +65,10 @@ #define NVS_TYPE_SUBCH_RESP 133 /* same as SUBCH_REQ */ #define NVS_TYPE_TXTBL_NOTE 134 /* notification */ +/* Private data for primary/secondary processes */ +struct hn_nvs_process_priv { + struct rte_vmbus_device *vmbus_dev; +}; /* NVS message common header */ struct __rte_packed_begin hn_nvs_hdr { -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [patch v4 2/6] net/netvsc: introduce get_vmbus_device to get the vmbus device 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli 2025-04-07 22:44 ` [patch v4 1/6] net/netvsc: introduce private data for storing vmbus device for secondary process longli @ 2025-04-07 22:45 ` longli 2025-04-07 22:45 ` [patch v4 3/6] bus/vmbus: store UIO fd for secondary process longli ` (3 subsequent siblings) 5 siblings, 0 replies; 9+ messages in thread From: longli @ 2025-04-07 22:45 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> Introduce a function get the vmbus device from hn_data. For secondary process, the vmbus device is in eth_dev's private region. Signed-off-by: Long Li <longli@microsoft.com> --- drivers/net/netvsc/hn_nvs.c | 15 +++++++++++++++ drivers/net/netvsc/hn_nvs.h | 2 ++ 2 files changed, 17 insertions(+) diff --git a/drivers/net/netvsc/hn_nvs.c b/drivers/net/netvsc/hn_nvs.c index 7db82af9f3..fd20e3d06d 100644 --- a/drivers/net/netvsc/hn_nvs.c +++ b/drivers/net/netvsc/hn_nvs.c @@ -44,6 +44,21 @@ static const uint32_t hn_nvs_version[] = { NVS_VERSION_1 }; +struct rte_vmbus_device *get_vmbus_device(struct hn_data *hv) +{ + struct rte_vmbus_device *vmbus = hv->vmbus; + + /* For secondary process, vmbus is in the eth_dev private */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { + struct rte_eth_dev *dev = &rte_eth_devices[hv->port_id]; + struct hn_nvs_process_priv *process_priv = dev->process_private; + + vmbus = process_priv->vmbus_dev; + } + + return vmbus; +} + static int hn_nvs_req_send(struct hn_data *hv, void *req, uint32_t reqlen) { diff --git a/drivers/net/netvsc/hn_nvs.h b/drivers/net/netvsc/hn_nvs.h index 3950749359..0d8fe27b65 100644 --- a/drivers/net/netvsc/hn_nvs.h +++ b/drivers/net/netvsc/hn_nvs.h @@ -221,6 +221,8 @@ void hn_nvs_handle_vfassoc(struct rte_eth_dev *dev, const struct vmbus_chanpkt_hdr *hdr, const void *data); +struct rte_vmbus_device *get_vmbus_device(struct hn_data *hv); + static inline int hn_nvs_send(struct vmbus_channel *chan, uint16_t flags, void *nvs_msg, int nvs_msglen, uintptr_t sndc, -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [patch v4 3/6] bus/vmbus: store UIO fd for secondary process 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli 2025-04-07 22:44 ` [patch v4 1/6] net/netvsc: introduce private data for storing vmbus device for secondary process longli 2025-04-07 22:45 ` [patch v4 2/6] net/netvsc: introduce get_vmbus_device to get the vmbus device longli @ 2025-04-07 22:45 ` longli 2025-04-07 22:45 ` [patch v4 4/6] bus/vmbus: support channels without monitoring enabled longli ` (2 subsequent siblings) 5 siblings, 0 replies; 9+ messages in thread From: longli @ 2025-04-07 22:45 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> Secondary process will get access to vmbus device and this UIO fd for signaling hyperv host on channels without monitoring support. Signed-off-by: Long Li <longli@microsoft.com> --- drivers/bus/vmbus/vmbus_common_uio.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/bus/vmbus/vmbus_common_uio.c b/drivers/bus/vmbus/vmbus_common_uio.c index 4d4613513c..d55aee6537 100644 --- a/drivers/bus/vmbus/vmbus_common_uio.c +++ b/drivers/bus/vmbus/vmbus_common_uio.c @@ -86,8 +86,11 @@ vmbus_uio_map_secondary(struct rte_vmbus_device *dev) return -1; } - /* fd is not needed in secondary process, close it */ - close(fd); + if (rte_intr_fd_set(dev->intr_handle, fd)) + return -1; + + if (rte_intr_type_set(dev->intr_handle, RTE_INTR_HANDLE_UIO_INTX)) + return -1; /* Create and map primary channel */ if (vmbus_chan_create(dev, dev->relid, 0, @@ -256,7 +259,7 @@ vmbus_uio_unmap_resource(struct rte_vmbus_device *dev) /* free uio resource */ rte_free(uio_res); - /* close fd if in primary process */ + /* close fd */ if (rte_intr_fd_get(dev->intr_handle) >= 0) close(rte_intr_fd_get(dev->intr_handle)); -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [patch v4 4/6] bus/vmbus: support channels without monitoring enabled 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli ` (2 preceding siblings ...) 2025-04-07 22:45 ` [patch v4 3/6] bus/vmbus: store UIO fd for secondary process longli @ 2025-04-07 22:45 ` longli 2025-04-07 22:45 ` [patch v4 5/6] bus/vmbus: add rte_vmbus_device to all functions accessing vmbus longli 2025-04-07 22:45 ` [patch v4 6/6] bus/vmbus: set event for channel without monitoring support longli 5 siblings, 0 replies; 9+ messages in thread From: longli @ 2025-04-07 22:45 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> Hyperv host may offer channels without monitor enabled. The max monitor ID it supports is 128. Over those channels without monitor enabled, Hyperv does not send or receive large amount of data traffic and almost all the data traffic is going over the VF. Change the code to not fail on creating channels without monitor enabled. Use UINT8_MAX (256) to indicate this channel have no monitoring. Signed-off-by: Long Li <longli@microsoft.com> --- drivers/bus/vmbus/linux/vmbus_bus.c | 9 ++++++--- drivers/bus/vmbus/linux/vmbus_uio.c | 4 ++-- drivers/bus/vmbus/vmbus_channel.c | 3 +++ 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/bus/vmbus/linux/vmbus_bus.c b/drivers/bus/vmbus/linux/vmbus_bus.c index 01d8111b85..79fd3370b8 100644 --- a/drivers/bus/vmbus/linux/vmbus_bus.c +++ b/drivers/bus/vmbus/linux/vmbus_bus.c @@ -280,9 +280,12 @@ vmbus_scan_one(const char *name) /* get monitor id */ snprintf(filename, sizeof(filename), "%s/monitor_id", dirname); - if (eal_parse_sysfs_value(filename, &tmp) < 0) - goto error; - dev->monitor_id = tmp; + if (eal_parse_sysfs_value(filename, &tmp) >= 0) { + dev->monitor_id = tmp; + } else { + VMBUS_LOG(NOTICE, "monitor disabled on %s", name); + dev->monitor_id = UINT8_MAX; + } /* get numa node (if present) */ snprintf(filename, sizeof(filename), "%s/numa_node", diff --git a/drivers/bus/vmbus/linux/vmbus_uio.c b/drivers/bus/vmbus/linux/vmbus_uio.c index 26edef342d..33edc151f6 100644 --- a/drivers/bus/vmbus/linux/vmbus_uio.c +++ b/drivers/bus/vmbus/linux/vmbus_uio.c @@ -451,9 +451,9 @@ int vmbus_uio_get_subchan(struct vmbus_channel *primary, err = vmbus_uio_sysfs_read(subchan_path, "monitor_id", &monid, UINT8_MAX); if (err) { - VMBUS_LOG(NOTICE, "no monitor_id in %s:%s", + VMBUS_LOG(NOTICE, "no monitor_id in %s:%s use int mode", subchan_path, strerror(-err)); - goto fail; + monid = UINT8_MAX; } err = vmbus_chan_create(dev, relid, subid, monid, subchan); diff --git a/drivers/bus/vmbus/vmbus_channel.c b/drivers/bus/vmbus/vmbus_channel.c index 925c2aa081..d4b5ba1979 100644 --- a/drivers/bus/vmbus/vmbus_channel.c +++ b/drivers/bus/vmbus/vmbus_channel.c @@ -52,6 +52,9 @@ rte_vmbus_set_latency(const struct rte_vmbus_device *dev, const struct vmbus_channel *chan, uint32_t latency) { + if (chan->monitor_id == UINT8_MAX) + return; + uint32_t trig_idx = chan->monitor_id / VMBUS_MONTRIG_LEN; uint32_t trig_offs = chan->monitor_id % VMBUS_MONTRIG_LEN; -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [patch v4 5/6] bus/vmbus: add rte_vmbus_device to all functions accessing vmbus 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli ` (3 preceding siblings ...) 2025-04-07 22:45 ` [patch v4 4/6] bus/vmbus: support channels without monitoring enabled longli @ 2025-04-07 22:45 ` longli 2025-04-07 22:45 ` [patch v4 6/6] bus/vmbus: set event for channel without monitoring support longli 5 siblings, 0 replies; 9+ messages in thread From: longli @ 2025-04-07 22:45 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> The secondary process can access its vmbus device through device private region. Add and pass it on all call chains leading to vmbus code. Signed-off-by: Long Li <longli@microsoft.com> --- drivers/bus/vmbus/linux/vmbus_uio.c | 2 +- drivers/bus/vmbus/private.h | 2 +- drivers/bus/vmbus/rte_bus_vmbus.h | 16 ++++++++++----- drivers/bus/vmbus/vmbus_channel.c | 32 +++++++++++++++++------------ drivers/net/netvsc/hn_nvs.c | 18 ++++++++-------- drivers/net/netvsc/hn_nvs.h | 15 +++++++------- drivers/net/netvsc/hn_rndis.c | 11 +++++----- drivers/net/netvsc/hn_rxtx.c | 16 +++++++-------- 8 files changed, 63 insertions(+), 49 deletions(-) diff --git a/drivers/bus/vmbus/linux/vmbus_uio.c b/drivers/bus/vmbus/linux/vmbus_uio.c index 33edc151f6..8edec869ac 100644 --- a/drivers/bus/vmbus/linux/vmbus_uio.c +++ b/drivers/bus/vmbus/linux/vmbus_uio.c @@ -27,7 +27,7 @@ static void *vmbus_map_addr; /* Control interrupts */ -void vmbus_uio_irq_control(struct rte_vmbus_device *dev, int32_t onoff) +void vmbus_uio_irq_control(const struct rte_vmbus_device *dev, int32_t onoff) { if ((rte_intr_fd_get(dev->intr_handle) < 0) || write(rte_intr_fd_get(dev->intr_handle), &onoff, diff --git a/drivers/bus/vmbus/private.h b/drivers/bus/vmbus/private.h index abee1b29f0..25b8a27fcf 100644 --- a/drivers/bus/vmbus/private.h +++ b/drivers/bus/vmbus/private.h @@ -110,7 +110,7 @@ void vmbus_insert_device(struct rte_vmbus_device *exist_vmbus_dev, struct rte_vmbus_device *new_vmbus_dev); void vmbus_remove_device(struct rte_vmbus_device *vmbus_device); -void vmbus_uio_irq_control(struct rte_vmbus_device *dev, int32_t onoff); +void vmbus_uio_irq_control(const struct rte_vmbus_device *dev, int32_t onoff); int vmbus_uio_irq_read(struct rte_vmbus_device *dev); int vmbus_uio_map_resource(struct rte_vmbus_device *dev); diff --git a/drivers/bus/vmbus/rte_bus_vmbus.h b/drivers/bus/vmbus/rte_bus_vmbus.h index fd18bca73c..2e9898ed7f 100644 --- a/drivers/bus/vmbus/rte_bus_vmbus.h +++ b/drivers/bus/vmbus/rte_bus_vmbus.h @@ -176,7 +176,8 @@ bool rte_vmbus_chan_rx_empty(const struct vmbus_channel *channel); * * Sends data in buffer directly to hyper-v via the vmbus */ -int rte_vmbus_chan_send(struct vmbus_channel *channel, uint16_t type, +int rte_vmbus_chan_send(struct rte_vmbus_device *dev, + struct vmbus_channel *channel, uint16_t type, void *data, uint32_t dlen, uint64_t xact, uint32_t flags, bool *need_sig); @@ -189,7 +190,8 @@ int rte_vmbus_chan_send(struct vmbus_channel *channel, uint16_t type, * Used when batching multiple sends and only signaling host * after the last send. */ -void rte_vmbus_chan_signal_tx(const struct vmbus_channel *channel); +void rte_vmbus_chan_signal_tx(struct rte_vmbus_device *dev, + const struct vmbus_channel *channel); /* Structure for scatter/gather I/O */ struct iova_list { @@ -223,7 +225,8 @@ struct iova_list { * * Sends data in buffer directly to hyper-v via the vmbus */ -int rte_vmbus_chan_send_sglist(struct vmbus_channel *channel, +int rte_vmbus_chan_send_sglist(struct rte_vmbus_device *dev, + struct vmbus_channel *channel, struct vmbus_gpa gpa[], uint32_t gpacnt, void *data, uint32_t dlen, uint64_t xact, bool *need_sig); @@ -243,7 +246,8 @@ int rte_vmbus_chan_send_sglist(struct vmbus_channel *channel, * On success, returns 0 * On failure, returns negative errno. */ -int rte_vmbus_chan_recv(struct vmbus_channel *chan, +int rte_vmbus_chan_recv(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, void *data, uint32_t *len, uint64_t *request_id); @@ -273,7 +277,9 @@ int rte_vmbus_chan_recv_raw(struct vmbus_channel *chan, * @param bytes_read * Number of bytes read since last signal */ -void rte_vmbus_chan_signal_read(struct vmbus_channel *chan, uint32_t bytes_read); +void rte_vmbus_chan_signal_read(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, + uint32_t bytes_read); /** * Determine sub channel index of the given channel diff --git a/drivers/bus/vmbus/vmbus_channel.c b/drivers/bus/vmbus/vmbus_channel.c index d4b5ba1979..bccef168d3 100644 --- a/drivers/bus/vmbus/vmbus_channel.c +++ b/drivers/bus/vmbus/vmbus_channel.c @@ -39,7 +39,8 @@ vmbus_set_monitor(const struct vmbus_channel *channel, uint32_t monitor_id) } static void -vmbus_set_event(const struct vmbus_channel *chan) +vmbus_set_event(struct rte_vmbus_device *dev __rte_unused, + const struct vmbus_channel *chan) { vmbus_set_monitor(chan, chan->monitor_id); } @@ -80,7 +81,7 @@ rte_vmbus_set_latency(const struct rte_vmbus_device *dev, * Can't do a hypercall from userspace. */ void -rte_vmbus_chan_signal_tx(const struct vmbus_channel *chan) +rte_vmbus_chan_signal_tx(struct rte_vmbus_device *dev, const struct vmbus_channel *chan) { const struct vmbus_br *tbr = &chan->txbr; @@ -91,14 +92,15 @@ rte_vmbus_chan_signal_tx(const struct vmbus_channel *chan) if (tbr->vbr->imask) return; - vmbus_set_event(chan); + vmbus_set_event(dev, chan); } /* Do a simple send directly using transmit ring. */ -int rte_vmbus_chan_send(struct vmbus_channel *chan, uint16_t type, - void *data, uint32_t dlen, - uint64_t xactid, uint32_t flags, bool *need_sig) +int rte_vmbus_chan_send(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, uint16_t type, void *data, + uint32_t dlen, uint64_t xactid, uint32_t flags, + bool *need_sig) { struct vmbus_chanpkt pkt; unsigned int pktlen, pad_pktlen; @@ -134,12 +136,13 @@ int rte_vmbus_chan_send(struct vmbus_channel *chan, uint16_t type, if (need_sig) *need_sig |= send_evt; else if (error == 0 && send_evt) - rte_vmbus_chan_signal_tx(chan); + rte_vmbus_chan_signal_tx(dev, chan); return error; } /* Do a scatter/gather send where the descriptor points to data. */ -int rte_vmbus_chan_send_sglist(struct vmbus_channel *chan, +int rte_vmbus_chan_send_sglist(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, struct vmbus_gpa sg[], uint32_t sglen, void *data, uint32_t dlen, uint64_t xactid, bool *need_sig) @@ -178,7 +181,7 @@ int rte_vmbus_chan_send_sglist(struct vmbus_channel *chan, if (need_sig) *need_sig |= send_evt; else if (error == 0 && send_evt) - rte_vmbus_chan_signal_tx(chan); + rte_vmbus_chan_signal_tx(dev, chan); return error; } @@ -191,7 +194,9 @@ bool rte_vmbus_chan_rx_empty(const struct vmbus_channel *channel) } /* Signal host after reading N bytes */ -void rte_vmbus_chan_signal_read(struct vmbus_channel *chan, uint32_t bytes_read) +void rte_vmbus_chan_signal_read(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, + uint32_t bytes_read) { struct vmbus_br *rbr = &chan->rxbr; uint32_t write_sz, pending_sz; @@ -218,10 +223,11 @@ void rte_vmbus_chan_signal_read(struct vmbus_channel *chan, uint32_t bytes_read) if (write_sz <= pending_sz) return; - vmbus_set_event(chan); + vmbus_set_event(dev, chan); } -int rte_vmbus_chan_recv(struct vmbus_channel *chan, void *data, uint32_t *len, +int rte_vmbus_chan_recv(struct rte_vmbus_device *dev, + struct vmbus_channel *chan, void *data, uint32_t *len, uint64_t *request_id) { struct vmbus_chanpkt_hdr pkt; @@ -263,7 +269,7 @@ int rte_vmbus_chan_recv(struct vmbus_channel *chan, void *data, uint32_t *len, if (error) return error; - rte_vmbus_chan_signal_read(chan, dlen + hlen + sizeof(uint64_t)); + rte_vmbus_chan_signal_read(dev, chan, dlen + hlen + sizeof(uint64_t)); return 0; } diff --git a/drivers/net/netvsc/hn_nvs.c b/drivers/net/netvsc/hn_nvs.c index fd20e3d06d..0b7e6c0264 100644 --- a/drivers/net/netvsc/hn_nvs.c +++ b/drivers/net/netvsc/hn_nvs.c @@ -62,7 +62,7 @@ struct rte_vmbus_device *get_vmbus_device(struct hn_data *hv) static int hn_nvs_req_send(struct hn_data *hv, void *req, uint32_t reqlen) { - return rte_vmbus_chan_send(hn_primary_chan(hv), + return rte_vmbus_chan_send(get_vmbus_device(hv), hn_primary_chan(hv), VMBUS_CHANPKT_TYPE_INBAND, req, reqlen, 0, VMBUS_CHANPKT_FLAG_NONE, NULL); @@ -82,8 +82,8 @@ __hn_nvs_execute(struct hn_data *hv, int ret; /* Send request to ring buffer */ - ret = rte_vmbus_chan_send(chan, VMBUS_CHANPKT_TYPE_INBAND, - req, reqlen, 0, + ret = rte_vmbus_chan_send(get_vmbus_device(hv), chan, + VMBUS_CHANPKT_TYPE_INBAND, req, reqlen, 0, VMBUS_CHANPKT_FLAG_RC, NULL); if (ret) { @@ -93,7 +93,7 @@ __hn_nvs_execute(struct hn_data *hv, retry: len = sizeof(buffer); - ret = rte_vmbus_chan_recv(chan, buffer, &len, &xactid); + ret = rte_vmbus_chan_recv(get_vmbus_device(hv), chan, buffer, &len, &xactid); if (ret == -EAGAIN) { rte_delay_us(HN_CHAN_INTERVAL_US); goto retry; @@ -114,7 +114,7 @@ __hn_nvs_execute(struct hn_data *hv, /* Silently drop received packets while waiting for response */ switch (hdr->type) { case NVS_TYPE_RNDIS: - hn_nvs_ack_rxbuf(chan, xactid); + hn_nvs_ack_rxbuf(hv, chan, xactid); /* fallthrough */ case NVS_TYPE_TXTBL_NOTE: @@ -518,7 +518,7 @@ hn_nvs_detach(struct hn_data *hv __rte_unused) * so that this RXBUF can be recycled by the hypervisor. */ void -hn_nvs_ack_rxbuf(struct vmbus_channel *chan, uint64_t tid) +hn_nvs_ack_rxbuf(struct hn_data *hv, struct vmbus_channel *chan, uint64_t tid) { unsigned int retries = 0; struct hn_nvs_rndis_ack ack = { @@ -530,9 +530,9 @@ hn_nvs_ack_rxbuf(struct vmbus_channel *chan, uint64_t tid) PMD_RX_LOG(DEBUG, "ack RX id %" PRIu64, tid); again: - error = rte_vmbus_chan_send(chan, VMBUS_CHANPKT_TYPE_COMP, - &ack, sizeof(ack), tid, - VMBUS_CHANPKT_FLAG_NONE, NULL); + error = rte_vmbus_chan_send(get_vmbus_device(hv), chan, + VMBUS_CHANPKT_TYPE_COMP, &ack, sizeof(ack), + tid, VMBUS_CHANPKT_FLAG_NONE, NULL); if (error == 0) return; diff --git a/drivers/net/netvsc/hn_nvs.h b/drivers/net/netvsc/hn_nvs.h index 0d8fe27b65..b7691a5dc2 100644 --- a/drivers/net/netvsc/hn_nvs.h +++ b/drivers/net/netvsc/hn_nvs.h @@ -214,7 +214,8 @@ struct __rte_packed_begin hn_nvs_rndis_ack { int hn_nvs_attach(struct hn_data *hv, unsigned int mtu); void hn_nvs_detach(struct hn_data *hv); -void hn_nvs_ack_rxbuf(struct vmbus_channel *chan, uint64_t tid); +void hn_nvs_ack_rxbuf(struct hn_data *hv, + struct vmbus_channel *chan, uint64_t tid); int hn_nvs_alloc_subchans(struct hn_data *hv, uint32_t *nsubch); int hn_nvs_set_datapath(struct hn_data *hv, uint32_t path); void hn_nvs_handle_vfassoc(struct rte_eth_dev *dev, @@ -224,21 +225,21 @@ void hn_nvs_handle_vfassoc(struct rte_eth_dev *dev, struct rte_vmbus_device *get_vmbus_device(struct hn_data *hv); static inline int -hn_nvs_send(struct vmbus_channel *chan, uint16_t flags, - void *nvs_msg, int nvs_msglen, uintptr_t sndc, +hn_nvs_send(struct hn_data *hv, struct vmbus_channel *chan, + uint16_t flags, void *nvs_msg, int nvs_msglen, uintptr_t sndc, bool *need_sig) { - return rte_vmbus_chan_send(chan, VMBUS_CHANPKT_TYPE_INBAND, + return rte_vmbus_chan_send(get_vmbus_device(hv), chan, VMBUS_CHANPKT_TYPE_INBAND, nvs_msg, nvs_msglen, (uint64_t)sndc, flags, need_sig); } static inline int -hn_nvs_send_sglist(struct vmbus_channel *chan, +hn_nvs_send_sglist(struct hn_data *hv, struct vmbus_channel *chan, struct vmbus_gpa sg[], unsigned int sglen, void *nvs_msg, int nvs_msglen, uintptr_t sndc, bool *need_sig) { - return rte_vmbus_chan_send_sglist(chan, sg, sglen, nvs_msg, nvs_msglen, - (uint64_t)sndc, need_sig); + return rte_vmbus_chan_send_sglist(get_vmbus_device(hv), chan, sg, sglen, nvs_msg, + nvs_msglen, (uint64_t)sndc, need_sig); } diff --git a/drivers/net/netvsc/hn_rndis.c b/drivers/net/netvsc/hn_rndis.c index b2dae6474a..12bea33e37 100644 --- a/drivers/net/netvsc/hn_rndis.c +++ b/drivers/net/netvsc/hn_rndis.c @@ -246,8 +246,9 @@ void hn_rndis_dump(const void *buf) } #endif -static int hn_nvs_send_rndis_ctrl(struct vmbus_channel *chan, - const void *req, uint32_t reqlen) +static int hn_nvs_send_rndis_ctrl(struct hn_data *hv, + struct vmbus_channel *chan, const void *req, + uint32_t reqlen) { struct hn_nvs_rndis nvs_rndis = { @@ -282,8 +283,8 @@ static int hn_nvs_send_rndis_ctrl(struct vmbus_channel *chan, hn_rndis_dump(req); - return hn_nvs_send_sglist(chan, &sg, 1, - &nvs_rndis, sizeof(nvs_rndis), 0U, NULL); + return hn_nvs_send_sglist(hv, chan, &sg, 1, &nvs_rndis, + sizeof(nvs_rndis), 0U, NULL); } /* @@ -394,7 +395,7 @@ static int hn_rndis_exec1(struct hn_data *hv, return -EBUSY; } - error = hn_nvs_send_rndis_ctrl(chan, req, reqlen); + error = hn_nvs_send_rndis_ctrl(hv, chan, req, reqlen); if (error) { PMD_DRV_LOG(ERR, "RNDIS ctrl send failed: %d", error); return error; diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 9d3948e03d..ffa6937ce2 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -556,9 +556,10 @@ static void hn_rx_buf_free_cb(void *buf __rte_unused, void *opaque) { struct hn_rx_bufinfo *rxb = opaque; struct hn_rx_queue *rxq = rxb->rxq; + struct hn_data *hv = rxq->hv; rte_atomic32_dec(&rxq->rxbuf_outstanding); - hn_nvs_ack_rxbuf(rxb->chan, rxb->xactid); + hn_nvs_ack_rxbuf(hv, rxb->chan, rxb->xactid); } static struct hn_rx_bufinfo *hn_rx_buf_init(struct hn_rx_queue *rxq, @@ -869,7 +870,7 @@ hn_nvs_handle_rxbuf(struct rte_eth_dev *dev, /* Send ACK now if external mbuf not used */ if (rte_mbuf_ext_refcnt_update(&rxb->shinfo, -1) == 0) - hn_nvs_ack_rxbuf(rxb->chan, rxb->xactid); + hn_nvs_ack_rxbuf(hv, rxb->chan, rxb->xactid); } /* @@ -1175,7 +1176,7 @@ uint32_t hn_process_events(struct hn_data *hv, uint16_t queue_id, } if (bytes_read > 0) - rte_vmbus_chan_signal_read(rxq->chan, bytes_read); + rte_vmbus_chan_signal_read(get_vmbus_device(hv), rxq->chan, bytes_read); rte_spinlock_unlock(&rxq->ring_lock); @@ -1232,7 +1233,7 @@ static int hn_flush_txagg(struct hn_tx_queue *txq, bool *need_sig) PMD_TX_LOG(DEBUG, "port %u:%u tx %u size %u", txq->port_id, txq->queue_id, txd->chim_index, txd->chim_size); - ret = hn_nvs_send(txq->chan, VMBUS_CHANPKT_FLAG_RC, + ret = hn_nvs_send(txq->hv, txq->chan, VMBUS_CHANPKT_FLAG_RC, &rndis, sizeof(rndis), (uintptr_t)txd, need_sig); if (likely(ret == 0)) @@ -1511,9 +1512,8 @@ static int hn_xmit_sg(struct hn_tx_queue *txq, txq->port_id, txq->queue_id, txd->chim_index, segs, nvs_rndis.chim_sz); - return hn_nvs_send_sglist(txq->chan, sg, segs, - &nvs_rndis, sizeof(nvs_rndis), - (uintptr_t)txd, need_sig); + return hn_nvs_send_sglist(txq->hv, txq->chan, sg, segs, &nvs_rndis, + sizeof(nvs_rndis), (uintptr_t)txd, need_sig); } uint16_t @@ -1641,7 +1641,7 @@ hn_xmit_pkts(void *ptxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) fail: if (need_sig) - rte_vmbus_chan_signal_tx(txq->chan); + rte_vmbus_chan_signal_tx(get_vmbus_device(hv), txq->chan); return nb_tx; } -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [patch v4 6/6] bus/vmbus: set event for channel without monitoring support 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli ` (4 preceding siblings ...) 2025-04-07 22:45 ` [patch v4 5/6] bus/vmbus: add rte_vmbus_device to all functions accessing vmbus longli @ 2025-04-07 22:45 ` longli 2025-04-08 14:18 ` Stephen Hemminger 5 siblings, 1 reply; 9+ messages in thread From: longli @ 2025-04-07 22:45 UTC (permalink / raw) To: Stephen Hemminger, Wei Hu; +Cc: dev, Long Li From: Long Li <longli@microsoft.com> For vmbus channels without monitoring support, use kernel UIO interface to indicate packet through interrupt page and UIO file handle. Signed-off-by: Long Li <longli@microsoft.com> --- Changes: v4: replace RTE_ATOMIC(uint32_t) * with RTE_ATOMIC(uint32_t *) for declaring variable drivers/bus/vmbus/vmbus_channel.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/drivers/bus/vmbus/vmbus_channel.c b/drivers/bus/vmbus/vmbus_channel.c index bccef168d3..96ac3ba6d6 100644 --- a/drivers/bus/vmbus/vmbus_channel.c +++ b/drivers/bus/vmbus/vmbus_channel.c @@ -19,15 +19,28 @@ #include "private.h" static inline void -vmbus_sync_set_bit(volatile RTE_ATOMIC(uint32_t) *addr, uint32_t mask) +vmbus_sync_set_bit(volatile RTE_ATOMIC(uint32_t *) addr, uint32_t mask) { rte_atomic_fetch_or_explicit(addr, mask, rte_memory_order_seq_cst); } +static inline void +vmbus_send_interrupt(const struct rte_vmbus_device *dev, uint32_t relid) +{ + RTE_ATOMIC(uint32_t *) int_addr; + uint32_t int_mask; + + int_addr = dev->int_page + relid / 32; + int_mask = 1u << (relid % 32); + vmbus_sync_set_bit(int_addr, int_mask); + + vmbus_uio_irq_control(dev, 1); +} + static inline void vmbus_set_monitor(const struct vmbus_channel *channel, uint32_t monitor_id) { - RTE_ATOMIC(uint32_t) *monitor_addr; + RTE_ATOMIC(uint32_t *) monitor_addr; uint32_t monitor_mask; unsigned int trigger_index; @@ -39,10 +52,13 @@ vmbus_set_monitor(const struct vmbus_channel *channel, uint32_t monitor_id) } static void -vmbus_set_event(struct rte_vmbus_device *dev __rte_unused, - const struct vmbus_channel *chan) +vmbus_set_event(struct rte_vmbus_device *dev, const struct vmbus_channel *chan) { - vmbus_set_monitor(chan, chan->monitor_id); + /* Use monitored bit if supported, otherwise use interrupt/Hypercall */ + if (chan->monitor_id != UINT8_MAX) + vmbus_set_monitor(chan, chan->monitor_id); + else + vmbus_send_interrupt(dev, chan->relid); } /* -- 2.34.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [patch v4 6/6] bus/vmbus: set event for channel without monitoring support 2025-04-07 22:45 ` [patch v4 6/6] bus/vmbus: set event for channel without monitoring support longli @ 2025-04-08 14:18 ` Stephen Hemminger 2025-04-08 21:55 ` [EXTERNAL] " Long Li 0 siblings, 1 reply; 9+ messages in thread From: Stephen Hemminger @ 2025-04-08 14:18 UTC (permalink / raw) To: longli; +Cc: Wei Hu, dev, Long Li On Mon, 7 Apr 2025 15:45:04 -0700 longli@linuxonhyperv.com wrote: > From: Long Li <longli@microsoft.com> > > For vmbus channels without monitoring support, use kernel UIO interface > to indicate packet through interrupt page and UIO file handle. > > Signed-off-by: Long Li <longli@microsoft.com> Something is wrong, no longer applies cleanly. $ git am ~/Downloads/bundle-1421-vmbus.mbox Applying: net/netvsc: introduce private data for storing vmbus device for secondary process Applying: net/netvsc: introduce get_vmbus_device to get the vmbus device Applying: bus/vmbus: store UIO fd for secondary process Applying: bus/vmbus: support channels without monitoring enabled Applying: bus/vmbus: add rte_vmbus_device to all functions accessing vmbus error: patch failed: drivers/bus/vmbus/vmbus_channel.c:80 error: drivers/bus/vmbus/vmbus_channel.c: patch does not apply Patch failed at 0005 bus/vmbus: add rte_vmbus_device to all functions accessing vmbus ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [EXTERNAL] Re: [patch v4 6/6] bus/vmbus: set event for channel without monitoring support 2025-04-08 14:18 ` Stephen Hemminger @ 2025-04-08 21:55 ` Long Li 0 siblings, 0 replies; 9+ messages in thread From: Long Li @ 2025-04-08 21:55 UTC (permalink / raw) To: Stephen Hemminger, longli; +Cc: Wei Hu, dev > Subject: [EXTERNAL] Re: [patch v4 6/6] bus/vmbus: set event for channel without > monitoring support > > On Mon, 7 Apr 2025 15:45:04 -0700 > longli@linuxonhyperv.com wrote: > > > From: Long Li <longli@microsoft.com> > > > > For vmbus channels without monitoring support, use kernel UIO > > interface to indicate packet through interrupt page and UIO file handle. > > > > Signed-off-by: Long Li <longli@microsoft.com> > > Something is wrong, no longer applies cleanly. > $ git am ~/Downloads/bundle-1421-vmbus.mbox > Applying: net/netvsc: introduce private data for storing vmbus device for > secondary process > Applying: net/netvsc: introduce get_vmbus_device to get the vmbus device > Applying: bus/vmbus: store UIO fd for secondary process > Applying: bus/vmbus: support channels without monitoring enabled > Applying: bus/vmbus: add rte_vmbus_device to all functions accessing vmbus > error: patch failed: drivers/bus/vmbus/vmbus_channel.c:80 > error: drivers/bus/vmbus/vmbus_channel.c: patch does not apply Patch failed at > 0005 bus/vmbus: add rte_vmbus_device to all functions accessing vmbus I rebased the patch to net-next and sent v5. Thanks, Long ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-04-08 21:55 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-04-07 22:44 [patch v4 0/6] Support VMBUS channels without monitoring enabled longli 2025-04-07 22:44 ` [patch v4 1/6] net/netvsc: introduce private data for storing vmbus device for secondary process longli 2025-04-07 22:45 ` [patch v4 2/6] net/netvsc: introduce get_vmbus_device to get the vmbus device longli 2025-04-07 22:45 ` [patch v4 3/6] bus/vmbus: store UIO fd for secondary process longli 2025-04-07 22:45 ` [patch v4 4/6] bus/vmbus: support channels without monitoring enabled longli 2025-04-07 22:45 ` [patch v4 5/6] bus/vmbus: add rte_vmbus_device to all functions accessing vmbus longli 2025-04-07 22:45 ` [patch v4 6/6] bus/vmbus: set event for channel without monitoring support longli 2025-04-08 14:18 ` Stephen Hemminger 2025-04-08 21:55 ` [EXTERNAL] " Long Li
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).