* [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP [not found] <20220825024425.10534-1-lihuisong@huawei.com> @ 2022-09-15 12:45 ` Huisong Li 2022-09-15 12:45 ` [PATCH V2 1/6] bus/pci: fix a segfault when call callback Huisong Li ` (5 more replies) 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (6 subsequent siblings) 7 siblings, 6 replies; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong This patchset do some bugfix and add attach and detach port for MP. --- -v2: resend due to CI unexplained failure. Huisong Li (6): bus/pci: fix a segfault when call callback bus/vdev: fix a segfault when call callback ethdev: fix push new event app/testpmd: check the validity of the port app/testpmd: support attach and detach port for MP app/testpmd: stop packet forwarding in new and destroy event app/test-pmd/testpmd.c | 85 +++++++++++++++++---------- app/test-pmd/testpmd.h | 1 - drivers/bus/pci/pci_common.c | 13 +++- drivers/bus/vdev/vdev.c | 11 +++- drivers/net/bonding/bonding_testpmd.c | 1 - lib/ethdev/ethdev_driver.c | 2 +- 6 files changed, 75 insertions(+), 38 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 1/6] bus/pci: fix a segfault when call callback 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li @ 2022-09-15 12:45 ` Huisong Li 2022-10-10 19:49 ` Thomas Monjalon 2022-09-15 12:45 ` [PATCH V2 2/6] bus/vdev: " Huisong Li ` (4 subsequent siblings) 5 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong After the driver probe is executed, the callback in application will be called. The callback in application may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. If the rte_pci_device::device::driver pointer isn't pointed to rte_pci_driver::driver in rte_pci_probe_one_driver, a segfault will occur. For example, when ethdev driver probe completes, the callback in application call rte_eth_dev_info_get which use dev->device->driver->name. So rte_pci_device::device::driver should point to rte_pci_driver::driver before executing the driver probe. Fixes: c752998b5e2e ("pci: introduce library and driver") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/pci/pci_common.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 37ab879779..831a9cd8c7 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -265,11 +265,22 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dr->driver.name, dev->id.vendor_id, dev->id.device_id, loc->domain, loc->bus, loc->devid, loc->function, dev->device.numa_node); + + /* + * After the driver probe is executed, the callback in application will + * be called. The callback in application may call some APIs which use + * dev->device.driver to get some driver information. If the driver + * pointer isn't pointed to driver->driver here, a segfault will occur. + */ + if (!already_probed) + dev->device.driver = &dr->driver; + /* call the driver probe() function */ ret = dr->probe(dr, dev); if (already_probed) return ret; /* no rollback if already succeeded earlier */ if (ret) { + dev->device.driver = NULL; dev->driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and @@ -282,8 +293,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 1/6] bus/pci: fix a segfault when call callback 2022-09-15 12:45 ` [PATCH V2 1/6] bus/pci: fix a segfault when call callback Huisong Li @ 2022-10-10 19:49 ` Thomas Monjalon 2022-10-25 3:25 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Thomas Monjalon @ 2022-10-10 19:49 UTC (permalink / raw) To: Huisong Li Cc: dev, ferruh.yigit, andrew.rybchenko, huangdaode, david.marchand 15/09/2022 14:45, Huisong Li: > After the driver probe is executed, the callback in application will > be called. The callback in application may call some APIs which access the > rte_pci_driver::driver by the device::driver pointer to get driver > information. If the rte_pci_device::device::driver pointer isn't pointed to > rte_pci_driver::driver in rte_pci_probe_one_driver, a segfault will occur. > For example, when ethdev driver probe completes, the callback in > application call rte_eth_dev_info_get which use dev->device->driver->name. > So rte_pci_device::device::driver should point to rte_pci_driver::driver > before executing the driver probe. > > Fixes: c752998b5e2e ("pci: introduce library and driver") > Cc: stable@dpdk.org To be more precise, it is reverting 391797f04208 ("drivers/bus: move driver assignment to end of probing") dev->device.driver is used by rte_dev_is_probed(): int rte_dev_is_probed(const struct rte_device *dev) { /* The field driver should be set only when the probe is successful. */ return dev->driver != NULL; } And the field comment is clear in rte_device: const struct rte_driver *driver; /**< Driver assigned after probing */ That's why I am not enthusiastic about setting this pointer before probing. I understand it is more convenient to use this pointer in a probing callback. We need to check it is not breaking rte_dev_is_probed() usage. It may be OK if there is no parallel probing, and rte_dev_is_probed() is not called inside probing. At the very minimum, we need to update some comments in the code, to mention that the pointer is set before probing, and reset if probing failed. ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 1/6] bus/pci: fix a segfault when call callback 2022-10-10 19:49 ` Thomas Monjalon @ 2022-10-25 3:25 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2022-10-25 3:25 UTC (permalink / raw) To: Thomas Monjalon Cc: dev, ferruh.yigit, andrew.rybchenko, huangdaode, david.marchand Hi Thomas, I missed this e-mail, I'm sorry for late reply. 在 2022/10/11 3:49, Thomas Monjalon 写道: > 15/09/2022 14:45, Huisong Li: >> After the driver probe is executed, the callback in application will >> be called. The callback in application may call some APIs which access the >> rte_pci_driver::driver by the device::driver pointer to get driver >> information. If the rte_pci_device::device::driver pointer isn't pointed to >> rte_pci_driver::driver in rte_pci_probe_one_driver, a segfault will occur. >> For example, when ethdev driver probe completes, the callback in >> application call rte_eth_dev_info_get which use dev->device->driver->name. >> So rte_pci_device::device::driver should point to rte_pci_driver::driver >> before executing the driver probe. >> >> Fixes: c752998b5e2e ("pci: introduce library and driver") >> Cc: stable@dpdk.org > To be more precise, it is reverting > 391797f04208 ("drivers/bus: move driver assignment to end of probing") > > dev->device.driver is used by rte_dev_is_probed(): > > int > rte_dev_is_probed(const struct rte_device *dev) > { > /* The field driver should be set only when the probe is successful. */ > return dev->driver != NULL; > } > > And the field comment is clear in rte_device: > > const struct rte_driver *driver; /**< Driver assigned after probing */ > > That's why I am not enthusiastic about setting this pointer before probing. +1 > > I understand it is more convenient to use this pointer > in a probing callback. > We need to check it is not breaking rte_dev_is_probed() usage. > It may be OK if there is no parallel probing, > and rte_dev_is_probed() is not called inside probing. From all the places where it's used, first of all, it is a internal interface and only used when scan device on bus, remove device and hotplug device. In process of scanning or probing device, the core code checks whether the device is already probed by calling rte_dev_is_probed(). So I think it is ok for parallel probing(if exist) to set dev->device.driver before probing successful. The Concurrency between probing and removing should not be possible. The application removes a device after device probing successful. Currently, rte_dev_is_probed() isn't called inside probing. > > At the very minimum, we need to update some comments in the code, > to mention that the pointer is set before probing, > and reset if probing failed. If it is ok for you, I will update some comments and add this modification to other bus type in next version. > > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 2/6] bus/vdev: fix a segfault when call callback 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-09-15 12:45 ` [PATCH V2 1/6] bus/pci: fix a segfault when call callback Huisong Li @ 2022-09-15 12:45 ` Huisong Li 2022-09-15 12:45 ` [PATCH V2 3/6] ethdev: fix push new event Huisong Li ` (3 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong After the driver probe is executed, the callback in application will be called. And this callback may call some APIs which access the driver in struct rte_vdev_driver by the device::driver pointer to get some driver information. If the rte_vdev_device::device::driver pointer isn't pointed to the rte_vdev_driver::driver before executing driver probe, a segfault will occur. Fixes: e9d159c3d534 ("eal: allow probing a device again") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/vdev/vdev.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index a8d8b2327e..dea3937607 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -209,9 +209,16 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * After the driver probe is executed, the callback in application will + * be called. The callback in application may call some APIs which use + * dev->device.driver to get some driver information. If the driver + * pointer isn't pointed to driver->driver here, a segfault will occur. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 3/6] ethdev: fix push new event 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-09-15 12:45 ` [PATCH V2 1/6] bus/pci: fix a segfault when call callback Huisong Li 2022-09-15 12:45 ` [PATCH V2 2/6] bus/vdev: " Huisong Li @ 2022-09-15 12:45 ` Huisong Li 2022-09-27 10:49 ` Thomas Monjalon 2022-09-15 12:45 ` [PATCH V2 4/6] app/testpmd: check the validity of the port Huisong Li ` (2 subsequent siblings) 5 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong The 'state' in struct rte_eth_dev may be used to update some information when app receive these events. For example, when app receives a new event, app may get the socket id of this port by calling rte_eth_dev_socket_id to setup the attached port. The 'state' is used in rte_eth_dev_socket_id. If the state isn't modified to RTE_ETH_DEV_ATTACHED before pushing the new event, app will get the socket id failed. So this patch moves pushing event operation after the state updated. Fixes: 99a2dd955fba ("lib: remove librte_ prefix from directory names") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- lib/ethdev/ethdev_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index a285f213f0..a6616f072b 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -206,9 +206,9 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; + rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); } int -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 3/6] ethdev: fix push new event 2022-09-15 12:45 ` [PATCH V2 3/6] ethdev: fix push new event Huisong Li @ 2022-09-27 10:49 ` Thomas Monjalon 2022-10-08 4:09 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Thomas Monjalon @ 2022-09-27 10:49 UTC (permalink / raw) To: Huisong Li; +Cc: dev, ferruh.yigit, andrew.rybchenko, huangdaode 15/09/2022 14:45, Huisong Li: > The 'state' in struct rte_eth_dev may be used to update some information > when app receive these events. For example, when app receives a new event, > app may get the socket id of this port by calling rte_eth_dev_socket_id to > setup the attached port. The 'state' is used in rte_eth_dev_socket_id. > > If the state isn't modified to RTE_ETH_DEV_ATTACHED before pushing the new > event, app will get the socket id failed. So this patch moves pushing event > operation after the state updated. > > Fixes: 99a2dd955fba ("lib: remove librte_ prefix from directory names") > Cc: stable@dpdk.org > > Signed-off-by: Huisong Li <lihuisong@huawei.com> > --- > lib/ethdev/ethdev_driver.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c > index a285f213f0..a6616f072b 100644 > --- a/lib/ethdev/ethdev_driver.c > +++ b/lib/ethdev/ethdev_driver.c > @@ -206,9 +206,9 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) > if (rte_eal_process_type() == RTE_PROC_SECONDARY) > eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); > > - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); > > dev->state = RTE_ETH_DEV_ATTACHED; > + rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); > } As explained in the first patch, I don't think it is a good solution. We should not allow the port to be used until the end of probing. When RTE_ETH_EVENT_NEW is sent, the device is allocated but not ready for use. If an entity like failsafe decides to take ownership of the port, then the application should not consider it at all. For these reasons, we should limit which operations can be done during RTE_ETH_EVENT_NEW processing. That's why I've proposed creating a new state RTE_ETH_DEV_ALLOCATED, not sure why you didn't follow this advice. ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 3/6] ethdev: fix push new event 2022-09-27 10:49 ` Thomas Monjalon @ 2022-10-08 4:09 ` lihuisong (C) 2022-10-25 3:26 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2022-10-08 4:09 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev, ferruh.yigit, andrew.rybchenko, huangdaode 在 2022/9/27 18:49, Thomas Monjalon 写道: > 15/09/2022 14:45, Huisong Li: >> The 'state' in struct rte_eth_dev may be used to update some information >> when app receive these events. For example, when app receives a new event, >> app may get the socket id of this port by calling rte_eth_dev_socket_id to >> setup the attached port. The 'state' is used in rte_eth_dev_socket_id. >> >> If the state isn't modified to RTE_ETH_DEV_ATTACHED before pushing the new >> event, app will get the socket id failed. So this patch moves pushing event >> operation after the state updated. >> >> Fixes: 99a2dd955fba ("lib: remove librte_ prefix from directory names") >> Cc: stable@dpdk.org >> >> Signed-off-by: Huisong Li <lihuisong@huawei.com> >> --- >> lib/ethdev/ethdev_driver.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c >> index a285f213f0..a6616f072b 100644 >> --- a/lib/ethdev/ethdev_driver.c >> +++ b/lib/ethdev/ethdev_driver.c >> @@ -206,9 +206,9 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) >> if (rte_eal_process_type() == RTE_PROC_SECONDARY) >> eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); >> >> - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); >> >> dev->state = RTE_ETH_DEV_ATTACHED; >> + rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); >> } > As explained in the first patch, I don't think it is a good solution. > We should not allow the port to be used until the end of probing. > When RTE_ETH_EVENT_NEW is sent, the device is allocated but > not ready for use. If an entity like failsafe decides to take ownership > of the port, then the application should not consider it at all. > For these reasons, we should limit which operations can be done > during RTE_ETH_EVENT_NEW processing. > That's why I've proposed creating a new state RTE_ETH_DEV_ALLOCATED, > not sure why you didn't follow this advice. I want to put this problem to this case this patchset mentions, so as to have a better discussion. From the first patch, I know what you mean. But I still have some confusion: When a device taken by failsafe PMD push new event, all event callback will be called under this device. As you suggested, the device is a valid port if its state is ALLOCATED or ATTACHED. Now, the macro RTE_ETH_FOREACH_DEV uses the condition that device is valid(state is ALLOCATED or ATTACHED) and NOT owned. Even if we add the new ALLOCATED state, this device taken by failsafe is also a valid port in new event callback in application, and is in 'NO OWNER'. if event callback of application is called before one of failsafe PMD. As a result, application still can operate this device directly. Can we make sure that event callback of failsafe PMD is before one of application? > > > > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 3/6] ethdev: fix push new event 2022-10-08 4:09 ` lihuisong (C) @ 2022-10-25 3:26 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2022-10-25 3:26 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev, ferruh.yigit, andrew.rybchenko, huangdaode 在 2022/10/8 12:09, lihuisong (C) 写道: > > 在 2022/9/27 18:49, Thomas Monjalon 写道: >> 15/09/2022 14:45, Huisong Li: >>> The 'state' in struct rte_eth_dev may be used to update some >>> information >>> when app receive these events. For example, when app receives a new >>> event, >>> app may get the socket id of this port by calling >>> rte_eth_dev_socket_id to >>> setup the attached port. The 'state' is used in rte_eth_dev_socket_id. >>> >>> If the state isn't modified to RTE_ETH_DEV_ATTACHED before pushing >>> the new >>> event, app will get the socket id failed. So this patch moves >>> pushing event >>> operation after the state updated. >>> >>> Fixes: 99a2dd955fba ("lib: remove librte_ prefix from directory names") >>> Cc: stable@dpdk.org >>> >>> Signed-off-by: Huisong Li <lihuisong@huawei.com> >>> --- >>> lib/ethdev/ethdev_driver.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c >>> index a285f213f0..a6616f072b 100644 >>> --- a/lib/ethdev/ethdev_driver.c >>> +++ b/lib/ethdev/ethdev_driver.c >>> @@ -206,9 +206,9 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) >>> if (rte_eal_process_type() == RTE_PROC_SECONDARY) >>> eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, >>> dev); >>> - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); >>> dev->state = RTE_ETH_DEV_ATTACHED; >>> + rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); >>> } >> As explained in the first patch, I don't think it is a good solution. >> We should not allow the port to be used until the end of probing. >> When RTE_ETH_EVENT_NEW is sent, the device is allocated but >> not ready for use. If an entity like failsafe decides to take ownership >> of the port, then the application should not consider it at all. >> For these reasons, we should limit which operations can be done >> during RTE_ETH_EVENT_NEW processing. >> That's why I've proposed creating a new state RTE_ETH_DEV_ALLOCATED, >> not sure why you didn't follow this advice. > I want to put this problem to this case this patchset mentions, so as to > have a better discussion. From the first patch, I know what you mean. > But I still have some confusion: > When a device taken by failsafe PMD push new event, all event callback > will be called under this device. As you suggested, the device is a valid > port if its state is ALLOCATED or ATTACHED. Now, the macro > RTE_ETH_FOREACH_DEV > uses the condition that device is valid(state is ALLOCATED or ATTACHED) > and NOT owned. > > Even if we add the new ALLOCATED state, this device taken by failsafe is > also a valid port in new event callback in application, and is in 'NO > OWNER'. > if event callback of application is called before one of failsafe PMD. > As a result, application still can operate this device directly. > Can we make sure that event callback of failsafe PMD is before one of > application? Hi Thomas, Can you look at my confusion? >> >> >> >> >> . > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 4/6] app/testpmd: check the validity of the port 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li ` (2 preceding siblings ...) 2022-09-15 12:45 ` [PATCH V2 3/6] ethdev: fix push new event Huisong Li @ 2022-09-15 12:45 ` Huisong Li 2022-09-22 5:07 ` Singh, Aman Deep 2022-09-15 12:45 ` [PATCH V2 5/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-09-15 12:45 ` [PATCH V2 6/6] app/testpmd: stop packet forwarding in new and destroy event Huisong Li 5 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 77741fc41f..1605e34f35 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3673,14 +3673,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V2 4/6] app/testpmd: check the validity of the port 2022-09-15 12:45 ` [PATCH V2 4/6] app/testpmd: check the validity of the port Huisong Li @ 2022-09-22 5:07 ` Singh, Aman Deep 0 siblings, 0 replies; 95+ messages in thread From: Singh, Aman Deep @ 2022-09-22 5:07 UTC (permalink / raw) To: Huisong Li, dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode On 9/15/2022 6:15 PM, Huisong Li wrote: > This patch checks the validity of port id for all events in > 'eth_event_callback()'. > > Signed-off-by: Huisong Li <lihuisong@huawei.com>Acked-by: Aman Singh <aman.deep.singh@intel.com> > --- ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 5/6] app/testpmd: support attach and detach port for MP 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li ` (3 preceding siblings ...) 2022-09-15 12:45 ` [PATCH V2 4/6] app/testpmd: check the validity of the port Huisong Li @ 2022-09-15 12:45 ` Huisong Li 2022-09-15 12:45 ` [PATCH V2 6/6] app/testpmd: stop packet forwarding in new and destroy event Huisong Li 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong This patch supports attach and detach port in primary and secondary process. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> --- app/test-pmd/testpmd.c | 76 +++++++++++++++++---------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 47 insertions(+), 31 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 1605e34f35..10b330e33c 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3323,9 +3323,6 @@ reset_port(portid_t pid) void attach_port(char *identifier) { - portid_t pi; - struct rte_dev_iterator iterator; - printf("Attaching a new port...\n"); if (identifier == NULL) { @@ -3337,24 +3334,6 @@ attach_port(char *identifier) TESTPMD_LOG(ERR, "Failed to attach port %s\n", identifier); return; } - - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); - return; - } - - /* second attach mode: iterator */ - RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { - /* setup ports matching the devargs used for probing */ - if (port_is_forwarding(pi)) - continue; /* port was already attached before */ - setup_attached_port(pi); - } } static void @@ -3377,7 +3356,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3411,11 +3389,10 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + return; } @@ -3481,11 +3458,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3649,11 +3624,52 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void config_attached_port(portid_t port_id) +{ + struct rte_dev_iterator iterator; + struct rte_eth_dev_info dev_info; + portid_t pi; + int ret; + + /* first attach mode: event */ + if (setup_on_probe_event) { + /* new ports are detected on RTE_ETH_EVENT_NEW event */ + setup_attached_port(port_id); + return; + } + + ret = eth_dev_info_get_print_err(port_id, &dev_info); + if (ret != 0) { + TESTPMD_LOG(ERR, + "Failed to get device info for port %d, not detaching\n", + port_id); + return; + } + /* second attach mode: iterator */ + RTE_ETH_FOREACH_MATCHING_DEV(pi, dev_info.device->name, &iterator) { + /* setup ports matching the devargs used for probing */ + if (port_is_forwarding(port_id)) + continue; /* port was already attached before */ + setup_attached_port(port_id); + } +} + +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3678,8 +3694,7 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + config_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3690,6 +3705,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index ddf5e21849..22261f20a2 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -278,7 +278,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 3941f4cf23..ed1a278a44 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -763,7 +763,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, port_id, rte_strerror(-ret)); ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V2 6/6] app/testpmd: stop packet forwarding in new and destroy event 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li ` (4 preceding siblings ...) 2022-09-15 12:45 ` [PATCH V2 5/6] app/testpmd: support attach and detach port for MP Huisong Li @ 2022-09-15 12:45 ` Huisong Li 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-09-15 12:45 UTC (permalink / raw) To: dev; +Cc: thomas, ferruh.yigit, andrew.rybchenko, huangdaode, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 10b330e33c..faea6358f9 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3694,6 +3694,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); config_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: @@ -3703,6 +3705,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li @ 2022-12-06 6:45 ` Huisong Li 2022-12-06 6:45 ` [PATCH V3 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (4 more replies) 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (5 subsequent siblings) 7 siblings, 5 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v3: 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add some similar modifications for other bus type. 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 1/5] drivers/bus: restore driver assignment at front of probing 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li @ 2022-12-06 6:45 ` Huisong Li 2022-12-06 6:45 ` [PATCH V3 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (3 subsequent siblings) 4 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index ff1369353a..13cb3fe0f8 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index bb943b58b5..5f23446f41 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -293,13 +293,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index bc3a7f39fe..efaa1792e9 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 41bc07dde7..2e3f0f2e12 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 8d32d66504..feb1651984 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 2/5] ethdev: fix skip valid port in probing callback 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 6:45 ` [PATCH V3 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2022-12-06 6:45 ` Huisong Li 2022-12-06 6:45 ` [PATCH V3 3/5] app/testpmd: check the validity of the port Huisong Li ` (2 subsequent siblings) 4 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is RTE_ETH_DEV_UNUSED, this port will be skiped. Currently, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and to RTE_ETH_DEV_ATTACHED when definitely probed. And this port is valid if its device state is ALLOCATED or ATTACHED. Naturally, a lot of places had to be modified for this change. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index b3de490d36..7b08353ed5 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6003,7 +6003,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_in_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e55be8720e..949d8c1367 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3014,7 +3014,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_in_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..7868958dff 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_in_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_in_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_in_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 6a550cfc83..e04f88fe46 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1542,6 +1542,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is in used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_in_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 94b8fba5d7..362e5c2dec 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_in_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index 838b3a8f9f..a78d503e59 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_in_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 5d5e18db1e..a225141937 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -325,7 +325,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -372,7 +372,7 @@ int rte_eth_dev_is_valid_port(uint16_t port_id) { if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) return 0; else return 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index c129ca1eaf..459ad44021 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2000,7 +2000,9 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 17201fbe0f..66d70f1aac 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -327,4 +327,5 @@ INTERNAL { rte_eth_representor_id_get; rte_eth_switch_domain_alloc; rte_eth_switch_domain_free; + rte_eth_dev_in_used; }; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 3/5] app/testpmd: check the validity of the port 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 6:45 ` [PATCH V3 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2022-12-06 6:45 ` [PATCH V3 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2022-12-06 6:45 ` Huisong Li 2022-12-06 6:45 ` [PATCH V3 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2022-12-06 6:45 ` [PATCH V3 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 4 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 134d79a555..bc25703490 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3798,14 +3798,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 4/5] app/testpmd: add attach and detach port for multiple process 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (2 preceding siblings ...) 2022-12-06 6:45 ` [PATCH V3 3/5] app/testpmd: check the validity of the port Huisong Li @ 2022-12-06 6:45 ` Huisong Li 2022-12-06 6:45 ` [PATCH V3 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 4 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patch supports attach and detach port in primary and secondary process. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index bc25703490..2e6329c853 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3463,15 +3463,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3502,7 +3499,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3536,10 +3532,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3606,11 +3600,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3774,11 +3766,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3803,8 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3815,6 +3818,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 7d24d25970..080d3a1139 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -306,7 +306,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 9529e16fb6..9216271314 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -765,7 +765,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V3 5/5] app/testpmd: stop forwarding in new or destroy event 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (3 preceding siblings ...) 2022-12-06 6:45 ` [PATCH V3 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2022-12-06 6:45 ` Huisong Li 4 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 6:45 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 2e6329c853..746f07652a 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3806,6 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3816,6 +3818,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2022-12-06 9:26 ` [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (6 more replies) 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li ` (4 subsequent siblings) 7 siblings, 7 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v4: fix a misspelling. -v3: 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2022-12-06 9:26 ` [PATCH V4 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (5 subsequent siblings) 6 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index ff1369353a..13cb3fe0f8 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index bb943b58b5..5f23446f41 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -293,13 +293,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index bc3a7f39fe..efaa1792e9 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 41bc07dde7..2e3f0f2e12 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 8d32d66504..feb1651984 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2022-12-06 9:26 ` [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 2:44 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 12:51 UTC (permalink / raw) To: Huisong Li, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 12/6/2022 9:26 AM, Huisong Li wrote: > The driver assignment was moved back at the end of the device probing > because there is no something to use rte_driver during the phase of > probing. See commit 391797f04208 ("drivers/bus: move driver assignment > to end of probing") > > However, it is necessary for probing callback to reference rte_driver > before probing. For example, probing callback may call some APIs which > access the rte_pci_driver::driver by the device::driver pointer to get > driver information. In this case, a segment fault will occur in probing > callback if there is not this assignment. > Probing callback gets driver as parameter, so callback function can access it via 'drv->driver', is there a specific usecase that 'dev->device->driver' needs to be accessed explicitly? I assume this is related to coming patches that setting up device in testpmd event callback, but can you please clarify exact need. > Further, some comments in code need to be updated if we do that. The > driver pointer in rte_device is set before probing and needs to be reset > if probing failed. And rte_dev_is_probed can not be called inside probing. > > Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") > Cc: stable@dpdk.org > > Signed-off-by: Huisong Li <lihuisong@huawei.com> > --- > drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- > drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- > drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- > drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- > drivers/bus/pci/pci_common.c | 9 +++++++-- > drivers/bus/vdev/vdev.c | 10 ++++++++-- > drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- > 7 files changed, 52 insertions(+), 14 deletions(-) > > diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c > index ff1369353a..13cb3fe0f8 100644 > --- a/drivers/bus/auxiliary/auxiliary_common.c > +++ b/drivers/bus/auxiliary/auxiliary_common.c > @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, > } > > dev->driver = drv; > + /* > + * Reference rte_driver before probing so as to this pointer can > + * be used to get driver information in case of segment fault in > + * probing callback. > + */ > + dev->device.driver = &drv->driver; > > AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", > drv->driver.name, dev->name, dev->device.numa_node); > ret = drv->probe(drv, dev); > if (ret != 0) { > dev->driver = NULL; > + dev->device.driver = NULL; > rte_intr_instance_free(dev->intr_handle); > dev->intr_handle = NULL; > - } else { > - dev->device.driver = &drv->driver; > } > > return ret; > diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c > index e57159f5d8..f1b817e58c 100644 > --- a/drivers/bus/dpaa/dpaa_bus.c > +++ b/drivers/bus/dpaa/dpaa_bus.c > @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) > (dev->device.devargs && > dev->device.devargs->policy == RTE_DEV_BLOCKED)) > continue; > - > + /* > + * Reference rte_driver before probing so as to this > + * pointer can be used to get driver information in case > + * of segment fault in probing callback. > + */ > + dev->device.driver = &drv->driver; > if (probe_all || > (dev->device.devargs && > dev->device.devargs->policy == RTE_DEV_ALLOWED)) { > ret = drv->probe(drv, dev); > if (ret) { > + dev->device.driver = NULL; > DPAA_BUS_ERR("unable to probe:%s", > dev->name); > } else { > dev->driver = drv; > - dev->device.driver = &drv->driver; > } > } > break; > diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c > index 57bfb5111a..4bc0c6d3d4 100644 > --- a/drivers/bus/fslmc/fslmc_bus.c > +++ b/drivers/bus/fslmc/fslmc_bus.c > @@ -471,15 +471,21 @@ rte_fslmc_probe(void) > continue; > } > > + /* > + * Reference rte_driver before probing so as to this > + * pointer can be used to get driver information in case > + * of segment fault in probing callback. > + */ > + dev->device.driver = &drv->driver; > if (probe_all || > (dev->device.devargs && > dev->device.devargs->policy == RTE_DEV_ALLOWED)) { > ret = drv->probe(drv, dev); > if (ret) { > + dev->device.driver = NULL; > DPAA2_BUS_ERR("Unable to probe"); > } else { > dev->driver = drv; > - dev->device.driver = &drv->driver; > } > } > break; > diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c > index bb943b58b5..5f23446f41 100644 > --- a/drivers/bus/ifpga/ifpga_bus.c > +++ b/drivers/bus/ifpga/ifpga_bus.c > @@ -293,13 +293,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, > > /* reference driver structure */ > afu_dev->driver = drv; > + /* > + * Reference rte_driver before probing so as to this pointer can > + * be used to get driver information in case of segment fault in > + * probing callback. > + */ > + afu_dev->device.driver = &drv->driver; > > /* call the driver probe() function */ > ret = drv->probe(afu_dev); > - if (ret) > + if (ret) { > afu_dev->driver = NULL; > - else > - afu_dev->device.driver = &drv->driver; > + afu_dev->device.driver = NULL; > + } > > return ret; > } > diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c > index bc3a7f39fe..efaa1792e9 100644 > --- a/drivers/bus/pci/pci_common.c > +++ b/drivers/bus/pci/pci_common.c > @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, > return ret; > } > } > + /* > + * Reference rte_driver before probing so as to this pointer can > + * be used to get driver information in case of segment fault in > + * probing callback. > + */ > + dev->device.driver = &dr->driver; > } > > RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", > @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, > return ret; /* no rollback if already succeeded earlier */ > if (ret) { > dev->driver = NULL; > + dev->device.driver = NULL; > if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && > /* Don't unmap if device is unsupported and > * driver needs mapped resources. > @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, > dev->vfio_req_intr_handle = NULL; > rte_intr_instance_free(dev->intr_handle); > dev->intr_handle = NULL; > - } else { > - dev->device.driver = &dr->driver; > } > > return ret; > diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c > index 41bc07dde7..2e3f0f2e12 100644 > --- a/drivers/bus/vdev/vdev.c > +++ b/drivers/bus/vdev/vdev.c > @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) > return -1; > } > > + /* > + * Reference rte_driver before probing so as to this pointer can > + * be used to get driver information in case of segment fault in > + * probing callback. > + */ > + dev->device.driver = &driver->driver; > ret = driver->probe(dev); > - if (ret == 0) > - dev->device.driver = &driver->driver; > + if (ret != 0) > + dev->device.driver = NULL; > return ret; > } > > diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c > index 8d32d66504..feb1651984 100644 > --- a/drivers/bus/vmbus/vmbus_common.c > +++ b/drivers/bus/vmbus/vmbus_common.c > @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, > > /* reference driver structure */ > dev->driver = dr; > + /* > + * Reference rte_driver before probing so as to this pointer can > + * be used to get driver information in case of segment fault in > + * probing callback. > + */ > + dev->device.driver = &dr->driver; > > if (dev->device.numa_node < 0 && rte_socket_count() > 1) > VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); > @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, > ret = dr->probe(dr, dev); > if (ret) { > dev->driver = NULL; > + dev->device.driver = NULL; > rte_vmbus_unmap_device(dev); > - } else { > - dev->device.driver = &dr->driver; > } > > return ret; ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-01-11 12:51 ` Ferruh Yigit @ 2023-01-12 2:44 ` lihuisong (C) 2023-02-15 16:09 ` Ferruh Yigit 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-01-12 2:44 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen 在 2023/1/11 20:51, Ferruh Yigit 写道: > On 12/6/2022 9:26 AM, Huisong Li wrote: >> The driver assignment was moved back at the end of the device probing >> because there is no something to use rte_driver during the phase of >> probing. See commit 391797f04208 ("drivers/bus: move driver assignment >> to end of probing") >> >> However, it is necessary for probing callback to reference rte_driver >> before probing. For example, probing callback may call some APIs which >> access the rte_pci_driver::driver by the device::driver pointer to get >> driver information. In this case, a segment fault will occur in probing >> callback if there is not this assignment. >> > Probing callback gets driver as parameter, so callback function can > access it via 'drv->driver', is there a specific usecase that > 'dev->device->driver' needs to be accessed explicitly? > > I assume this is related to coming patches that setting up device in > testpmd event callback, but can you please clarify exact need. For example, rte_eth_dev_info_get is called in this event callback to get driver name. > >> Further, some comments in code need to be updated if we do that. The >> driver pointer in rte_device is set before probing and needs to be reset >> if probing failed. And rte_dev_is_probed can not be called inside probing. >> >> Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") >> Cc: stable@dpdk.org >> >> Signed-off-by: Huisong Li <lihuisong@huawei.com> >> --- >> drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- >> drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- >> drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- >> drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- >> drivers/bus/pci/pci_common.c | 9 +++++++-- >> drivers/bus/vdev/vdev.c | 10 ++++++++-- >> drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- >> 7 files changed, 52 insertions(+), 14 deletions(-) >> >> diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c >> index ff1369353a..13cb3fe0f8 100644 >> --- a/drivers/bus/auxiliary/auxiliary_common.c >> +++ b/drivers/bus/auxiliary/auxiliary_common.c >> @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, >> } >> >> dev->driver = drv; >> + /* >> + * Reference rte_driver before probing so as to this pointer can >> + * be used to get driver information in case of segment fault in >> + * probing callback. >> + */ >> + dev->device.driver = &drv->driver; >> >> AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", >> drv->driver.name, dev->name, dev->device.numa_node); >> ret = drv->probe(drv, dev); >> if (ret != 0) { >> dev->driver = NULL; >> + dev->device.driver = NULL; >> rte_intr_instance_free(dev->intr_handle); >> dev->intr_handle = NULL; >> - } else { >> - dev->device.driver = &drv->driver; >> } >> >> return ret; >> diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c >> index e57159f5d8..f1b817e58c 100644 >> --- a/drivers/bus/dpaa/dpaa_bus.c >> +++ b/drivers/bus/dpaa/dpaa_bus.c >> @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) >> (dev->device.devargs && >> dev->device.devargs->policy == RTE_DEV_BLOCKED)) >> continue; >> - >> + /* >> + * Reference rte_driver before probing so as to this >> + * pointer can be used to get driver information in case >> + * of segment fault in probing callback. >> + */ >> + dev->device.driver = &drv->driver; >> if (probe_all || >> (dev->device.devargs && >> dev->device.devargs->policy == RTE_DEV_ALLOWED)) { >> ret = drv->probe(drv, dev); >> if (ret) { >> + dev->device.driver = NULL; >> DPAA_BUS_ERR("unable to probe:%s", >> dev->name); >> } else { >> dev->driver = drv; >> - dev->device.driver = &drv->driver; >> } >> } >> break; >> diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c >> index 57bfb5111a..4bc0c6d3d4 100644 >> --- a/drivers/bus/fslmc/fslmc_bus.c >> +++ b/drivers/bus/fslmc/fslmc_bus.c >> @@ -471,15 +471,21 @@ rte_fslmc_probe(void) >> continue; >> } >> >> + /* >> + * Reference rte_driver before probing so as to this >> + * pointer can be used to get driver information in case >> + * of segment fault in probing callback. >> + */ >> + dev->device.driver = &drv->driver; >> if (probe_all || >> (dev->device.devargs && >> dev->device.devargs->policy == RTE_DEV_ALLOWED)) { >> ret = drv->probe(drv, dev); >> if (ret) { >> + dev->device.driver = NULL; >> DPAA2_BUS_ERR("Unable to probe"); >> } else { >> dev->driver = drv; >> - dev->device.driver = &drv->driver; >> } >> } >> break; >> diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c >> index bb943b58b5..5f23446f41 100644 >> --- a/drivers/bus/ifpga/ifpga_bus.c >> +++ b/drivers/bus/ifpga/ifpga_bus.c >> @@ -293,13 +293,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, >> >> /* reference driver structure */ >> afu_dev->driver = drv; >> + /* >> + * Reference rte_driver before probing so as to this pointer can >> + * be used to get driver information in case of segment fault in >> + * probing callback. >> + */ >> + afu_dev->device.driver = &drv->driver; >> >> /* call the driver probe() function */ >> ret = drv->probe(afu_dev); >> - if (ret) >> + if (ret) { >> afu_dev->driver = NULL; >> - else >> - afu_dev->device.driver = &drv->driver; >> + afu_dev->device.driver = NULL; >> + } >> >> return ret; >> } >> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c >> index bc3a7f39fe..efaa1792e9 100644 >> --- a/drivers/bus/pci/pci_common.c >> +++ b/drivers/bus/pci/pci_common.c >> @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, >> return ret; >> } >> } >> + /* >> + * Reference rte_driver before probing so as to this pointer can >> + * be used to get driver information in case of segment fault in >> + * probing callback. >> + */ >> + dev->device.driver = &dr->driver; >> } >> >> RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", >> @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, >> return ret; /* no rollback if already succeeded earlier */ >> if (ret) { >> dev->driver = NULL; >> + dev->device.driver = NULL; >> if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && >> /* Don't unmap if device is unsupported and >> * driver needs mapped resources. >> @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, >> dev->vfio_req_intr_handle = NULL; >> rte_intr_instance_free(dev->intr_handle); >> dev->intr_handle = NULL; >> - } else { >> - dev->device.driver = &dr->driver; >> } >> >> return ret; >> diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c >> index 41bc07dde7..2e3f0f2e12 100644 >> --- a/drivers/bus/vdev/vdev.c >> +++ b/drivers/bus/vdev/vdev.c >> @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) >> return -1; >> } >> >> + /* >> + * Reference rte_driver before probing so as to this pointer can >> + * be used to get driver information in case of segment fault in >> + * probing callback. >> + */ >> + dev->device.driver = &driver->driver; >> ret = driver->probe(dev); >> - if (ret == 0) >> - dev->device.driver = &driver->driver; >> + if (ret != 0) >> + dev->device.driver = NULL; >> return ret; >> } >> >> diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c >> index 8d32d66504..feb1651984 100644 >> --- a/drivers/bus/vmbus/vmbus_common.c >> +++ b/drivers/bus/vmbus/vmbus_common.c >> @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, >> >> /* reference driver structure */ >> dev->driver = dr; >> + /* >> + * Reference rte_driver before probing so as to this pointer can >> + * be used to get driver information in case of segment fault in >> + * probing callback. >> + */ >> + dev->device.driver = &dr->driver; >> >> if (dev->device.numa_node < 0 && rte_socket_count() > 1) >> VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); >> @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, >> ret = dr->probe(dr, dev); >> if (ret) { >> dev->driver = NULL; >> + dev->device.driver = NULL; >> rte_vmbus_unmap_device(dev); >> - } else { >> - dev->device.driver = &dr->driver; >> } >> >> return ret; > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-01-12 2:44 ` lihuisong (C) @ 2023-02-15 16:09 ` Ferruh Yigit 2023-02-28 2:21 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-02-15 16:09 UTC (permalink / raw) To: lihuisong (C), dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 1/12/2023 2:44 AM, lihuisong (C) wrote: > > 在 2023/1/11 20:51, Ferruh Yigit 写道: >> On 12/6/2022 9:26 AM, Huisong Li wrote: >>> The driver assignment was moved back at the end of the device probing >>> because there is no something to use rte_driver during the phase of >>> probing. See commit 391797f04208 ("drivers/bus: move driver assignment >>> to end of probing") >>> >>> However, it is necessary for probing callback to reference rte_driver >>> before probing. For example, probing callback may call some APIs which >>> access the rte_pci_driver::driver by the device::driver pointer to get >>> driver information. In this case, a segment fault will occur in probing >>> callback if there is not this assignment. >>> >> Probing callback gets driver as parameter, so callback function can >> access it via 'drv->driver', is there a specific usecase that >> 'dev->device->driver' needs to be accessed explicitly? >> >> I assume this is related to coming patches that setting up device in >> testpmd event callback, but can you please clarify exact need. > > For example, rte_eth_dev_info_get is called in this event callback to get > driver name. > Why 'rte_eth_dev_info_get()' is called in the event called at first place? This set updates multiple things to extend 'RTE_ETH_EVENT_NEW' event callback function support, like: - Assign device driver *before* probing completed, so that even callback can run 'rte_eth_dev_info_get()' - Add a new ethdev state so that port can be recognized as valid port in the even callback. - Stop forwarding implicitly in even callback in case event callback run while forwarding is on. All looks to me hack/complexity to make a specific case work, which is make secondary *testmp* application work with attached/detached device. And finally patch 4/5 adds port setup to testpmd event callback for this. I understand the intention, but I disagree with bus and ethdev level changes for this. Event callback may not be only way to share port attach/detach information between primary and secondary, there is a MP socket and 'rte_mp_handle' thread to handle communication between primary and secondary process, this should be able to use carrying device information, as far as I remember this is why it is introduced at first place. Did you consider using MP socket for your use case? Following is a sample usage: Primary: started as: sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 0-1 --log-level=*:debug -- -i --num-procs=2 --proc-id=0 `` testpmd> show port summary all Number of available ports: 0 Port MAC Address Name Driver Status Link testpmd> port attach net_null0 Attaching a new port... dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) vdev_probe_all_drivers(): Search driver to probe device net_null0 rte_pmd_null_probe(): Initializing pmd_null for net_null0 rte_pmd_null_probe(): Configure pmd_null: packet size is 64, packet copy is disabled eth_dev_null_create(): Creating null ethdev on numa socket 0 EAL: request: eal_dev_mp_request EAL: msg: bus_vdev_mp vdev_action(): send vdev, net_null0 EAL: sendmsg: bus_vdev_mp EAL: reply: bus_vdev_mp EAL: msg: eal_dev_mp_request Port 0 is attached. Now total ports is 1 Done testpmd> show port summary all Number of available ports: 1 Port MAC Address Name Driver Status Link 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps testpmd> port detach 0 Port was not closed Removing a device... EAL: request: eal_dev_mp_request EAL: msg: eal_dev_mp_request eth_dev_close(): Closing null ethdev on NUMA socket 0 Port 0 is closed Device is detached Now total ports is 0 Done testpmd> show port summary all Number of available ports: 0 Port MAC Address Name Driver Status Link testpmd> `` Secondary: started as: sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 2-3 --log-level=*:debug -- -i --num-procs=2 --proc-id=1 `` testpmd> show port summary all Number of available ports: 0 Port MAC Address Name Driver Status Link testpmd> EAL: msg: eal_dev_mp_request dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) EAL: request: bus_vdev_mp EAL: msg: bus_vdev_mp vdev_action(): receive vdev, net_null0 EAL: msg: bus_vdev_mp vdev_scan(): Received 1 vdevs vdev_probe_all_drivers(): Search driver to probe device net_null0 rte_pmd_null_probe(): Initializing pmd_null for net_null0 EAL: reply: eal_dev_mp_request testpmd> show port summary all Number of available ports: 1 Port MAC Address Name Driver Status Link 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps testpmd> EAL: msg: eal_dev_mp_request dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) eth_dev_close(): Closing null ethdev on NUMA socket 4294967295 Port 0 is closed EAL: reply: eal_dev_mp_request testpmd> show port summary all Number of available ports: 0 Port MAC Address Name Driver Status Link testpmd> `` ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-02-15 16:09 ` Ferruh Yigit @ 2023-02-28 2:21 ` lihuisong (C) 2023-06-06 16:12 ` Ferruh Yigit 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-02-28 2:21 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen [-- Attachment #1: Type: text/plain, Size: 6868 bytes --] 在 2023/2/16 0:09, Ferruh Yigit 写道: > On 1/12/2023 2:44 AM, lihuisong (C) wrote: >> 在 2023/1/11 20:51, Ferruh Yigit 写道: >>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>> The driver assignment was moved back at the end of the device probing >>>> because there is no something to use rte_driver during the phase of >>>> probing. See commit 391797f04208 ("drivers/bus: move driver assignment >>>> to end of probing") >>>> >>>> However, it is necessary for probing callback to reference rte_driver >>>> before probing. For example, probing callback may call some APIs which >>>> access the rte_pci_driver::driver by the device::driver pointer to get >>>> driver information. In this case, a segment fault will occur in probing >>>> callback if there is not this assignment. >>>> >>> Probing callback gets driver as parameter, so callback function can >>> access it via 'drv->driver', is there a specific usecase that >>> 'dev->device->driver' needs to be accessed explicitly? >>> >>> I assume this is related to coming patches that setting up device in >>> testpmd event callback, but can you please clarify exact need. >> For example, rte_eth_dev_info_get is called in this event callback to get >> driver name. >> Hi Ferruh, Sorry for the delay. I missed this email. Thanks for your review. > Why 'rte_eth_dev_info_get()' is called in the event called at first place? There is no limitation on rte_eth_dev_info_get() in event callback. The upper layer is entirely possible to use. After all, it is more convenient for app to use this pointer to get driver information in a probing callback. > > This set updates multiple things to extend 'RTE_ETH_EVENT_NEW' event > callback function support, like: > > - Assign device driver *before* probing completed, so that even callback > can run 'rte_eth_dev_info_get()' > > - Add a new ethdev state so that port can be recognized as valid port in > the even callback. Yes > > - Stop forwarding implicitly in even callback in case event callback run > while forwarding is on. But this is a modification for tesptmd to make it work well. > > > All looks to me hack/complexity to make a specific case work, which is > make secondary *testmp* application work with attached/detached device. It is not just a testpmd application case, but, I think, still exists in actual case. Because eal lib supports hotplugging device on primary and secondary process and the communication each other when attach or detach device. The reason why no one has ever put forward this question, I think it may be attributed to the fact that the scene is not or rarely tested. > > And finally patch 4/5 adds port setup to testpmd event callback for this. > > > I understand the intention, but I disagree with bus and ethdev level > changes for this. I'm just raising this issue, and we can discuss how to deal with it together.😁 > > > > Event callback may not be only way to share port attach/detach > information between primary and secondary, there is a MP socket and > 'rte_mp_handle' thread to handle communication between primary and > secondary process, this should be able to use carrying device > information, as far as I remember this is why it is introduced at first > place. > > Did you consider using MP socket for your use case? Actually, the probing event callback called in patch 4/5 is the result of the MP socket communication (please see hogplug_mp.c). > > > > Following is a sample usage: > > Primary: > started as: > sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 0-1 > --log-level=*:debug -- -i --num-procs=2 --proc-id=0 > > `` > testpmd> show port summary all > Number of available ports: 0 > Port MAC Address Name Driver Status Link > > testpmd> port attach net_null0 > Attaching a new port... > dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) > fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) > fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) > vdev_probe_all_drivers(): Search driver to probe device net_null0 > rte_pmd_null_probe(): Initializing pmd_null for net_null0 > rte_pmd_null_probe(): Configure pmd_null: packet size is 64, packet copy > is disabled > eth_dev_null_create(): Creating null ethdev on numa socket 0 > EAL: request: eal_dev_mp_request > EAL: msg: bus_vdev_mp > vdev_action(): send vdev, net_null0 > EAL: sendmsg: bus_vdev_mp > EAL: reply: bus_vdev_mp > EAL: msg: eal_dev_mp_request > Port 0 is attached. Now total ports is 1 > Done > > testpmd> show port summary all > Number of available ports: 1 > Port MAC Address Name Driver Status Link > 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps > > testpmd> port detach 0 > Port was not closed > Removing a device... > EAL: request: eal_dev_mp_request > EAL: msg: eal_dev_mp_request > eth_dev_close(): Closing null ethdev on NUMA socket 0 > Port 0 is closed > Device is detached > Now total ports is 0 Please note the log *"Now total ports is 0"*, which indicates the port number is updated if we detached device in this process. > Done > > testpmd> show port summary all > Number of available ports: 0 > Port MAC Address Name Driver Status Link > testpmd> > > `` > > Secondary: > started as: > sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 2-3 > --log-level=*:debug -- -i --num-procs=2 --proc-id=1 > > `` > testpmd> show port summary all > Number of available ports: 0 > Port MAC Address Name Driver Status Link > > testpmd> EAL: msg: eal_dev_mp_request > dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) > fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) > fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) > EAL: request: bus_vdev_mp > EAL: msg: bus_vdev_mp > vdev_action(): receive vdev, net_null0 > EAL: msg: bus_vdev_mp > vdev_scan(): Received 1 vdevs > vdev_probe_all_drivers(): Search driver to probe device net_null0 > rte_pmd_null_probe(): Initializing pmd_null for net_null0 > EAL: reply: eal_dev_mp_request > > testpmd> show port summary all > Number of available ports: 1 > Port MAC Address Name Driver Status Link > 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps > > testpmd> EAL: msg: eal_dev_mp_request > dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) > fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) > fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) > eth_dev_close(): Closing null ethdev on NUMA socket 4294967295 > Port 0 is closed > EAL: reply: eal_dev_mp_request But the port number in this process does not be updated after finishing the destroy event. That's the problem. > testpmd> show port summary all > Number of available ports: 0 > Port MAC Address Name Driver Status Link > testpmd> > `` > > > . [-- Attachment #2: Type: text/html, Size: 8946 bytes --] ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-02-28 2:21 ` lihuisong (C) @ 2023-06-06 16:12 ` Ferruh Yigit 2023-06-07 10:11 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-06-06 16:12 UTC (permalink / raw) To: lihuisong (C), dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 2/28/2023 2:21 AM, lihuisong (C) wrote: > > 在 2023/2/16 0:09, Ferruh Yigit 写道: >> On 1/12/2023 2:44 AM, lihuisong (C) wrote: >>> 在 2023/1/11 20:51, Ferruh Yigit 写道: >>>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>>> The driver assignment was moved back at the end of the device probing >>>>> because there is no something to use rte_driver during the phase of >>>>> probing. See commit 391797f04208 ("drivers/bus: move driver assignment >>>>> to end of probing") >>>>> >>>>> However, it is necessary for probing callback to reference rte_driver >>>>> before probing. For example, probing callback may call some APIs which >>>>> access the rte_pci_driver::driver by the device::driver pointer to get >>>>> driver information. In this case, a segment fault will occur in probing >>>>> callback if there is not this assignment. >>>>> >>>> Probing callback gets driver as parameter, so callback function can >>>> access it via 'drv->driver', is there a specific usecase that >>>> 'dev->device->driver' needs to be accessed explicitly? >>>> >>>> I assume this is related to coming patches that setting up device in >>>> testpmd event callback, but can you please clarify exact need. >>> For example, rte_eth_dev_info_get is called in this event callback to get >>> driver name. >>> > Hi Ferruh, > > Sorry for the delay. I missed this email. > Thanks for your review. >> Why 'rte_eth_dev_info_get()' is called in the event called at first place? > There is no limitation on rte_eth_dev_info_get() in event callback. > The upper layer is entirely possible to use. > After all, it is more convenient for app to use this pointer to get > driver information in a probing callback. > I think there is a logical limit in calling 'rte_eth_dev_info_get()' in NEW event callback, we may document this if it is missing. The NEW event callback is to let application do proper tasks before port addition finalized in the ethdev layer, but you are trying to call 'rte_eth_dev_info_get()' in the callback when it is not ready yet. >> This set updates multiple things to extend 'RTE_ETH_EVENT_NEW' event >> callback function support, like: >> >> - Assign device driver *before* probing completed, so that even callback >> can run 'rte_eth_dev_info_get()' >> >> - Add a new ethdev state so that port can be recognized as valid port in >> the even callback. > Yes >> - Stop forwarding implicitly in even callback in case event callback run >> while forwarding is on. > But this is a modification for tesptmd to make it work well. >> All looks to me hack/complexity to make a specific case work, which is >> make secondary *testmp* application work with attached/detached device. > It is not just a testpmd application case, but, I think, still exists in > actual case. > Because eal lib supports hotplugging device on primary and secondary > process and the communication each other when attach or detach device. > The reason why no one has ever put forward this question, I think it may > be attributed to the fact that the scene is not or rarely tested. >> And finally patch 4/5 adds port setup to testpmd event callback for this. >> >> >> I understand the intention, but I disagree with bus and ethdev level >> changes for this. > I'm just raising this issue, and we can discuss how to deal with it > together.😁 >> >> Event callback may not be only way to share port attach/detach >> information between primary and secondary, there is a MP socket and >> 'rte_mp_handle' thread to handle communication between primary and >> secondary process, this should be able to use carrying device >> information, as far as I remember this is why it is introduced at first >> place. >> >> Did you consider using MP socket for your use case? > Actually, the probing event callback called in patch 4/5 is the result > of the MP socket communication (please see hogplug_mp.c). >> >> Following is a sample usage: >> >> Primary: >> started as: >> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 0-1 >> --log-level=*:debug -- -i --num-procs=2 --proc-id=0 >> >> `` >> testpmd> show port summary all >> Number of available ports: 0 >> Port MAC Address Name Driver Status Link >> >> testpmd> port attach net_null0 >> Attaching a new port... >> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >> vdev_probe_all_drivers(): Search driver to probe device net_null0 >> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >> rte_pmd_null_probe(): Configure pmd_null: packet size is 64, packet copy >> is disabled >> eth_dev_null_create(): Creating null ethdev on numa socket 0 >> EAL: request: eal_dev_mp_request >> EAL: msg: bus_vdev_mp >> vdev_action(): send vdev, net_null0 >> EAL: sendmsg: bus_vdev_mp >> EAL: reply: bus_vdev_mp >> EAL: msg: eal_dev_mp_request >> Port 0 is attached. Now total ports is 1 >> Done >> >> testpmd> show port summary all >> Number of available ports: 1 >> Port MAC Address Name Driver Status Link >> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >> >> testpmd> port detach 0 >> Port was not closed >> Removing a device... >> EAL: request: eal_dev_mp_request >> EAL: msg: eal_dev_mp_request >> eth_dev_close(): Closing null ethdev on NUMA socket 0 >> Port 0 is closed >> Device is detached >> Now total ports is 0 > Please note the log *"Now total ports is 0"*, > which indicates the port number is updated if we detached device in this > process. > Yes, as expected. >> Done >> >> testpmd> show port summary all >> Number of available ports: 0 >> Port MAC Address Name Driver Status Link >> testpmd> >> >> `` >> >> Secondary: >> started as: >> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 2-3 >> --log-level=*:debug -- -i --num-procs=2 --proc-id=1 >> >> `` >> testpmd> show port summary all >> Number of available ports: 0 >> Port MAC Address Name Driver Status Link >> >> testpmd> EAL: msg: eal_dev_mp_request >> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >> EAL: request: bus_vdev_mp >> EAL: msg: bus_vdev_mp >> vdev_action(): receive vdev, net_null0 >> EAL: msg: bus_vdev_mp >> vdev_scan(): Received 1 vdevs >> vdev_probe_all_drivers(): Search driver to probe device net_null0 >> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >> EAL: reply: eal_dev_mp_request >> >> testpmd> show port summary all >> Number of available ports: 1 >> Port MAC Address Name Driver Status Link >> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >> >> testpmd> EAL: msg: eal_dev_mp_request >> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >> eth_dev_close(): Closing null ethdev on NUMA socket 4294967295 >> Port 0 is closed >> EAL: reply: eal_dev_mp_request > But the port number in this process does not be updated after finishing > the destroy event. > That's the problem. > Please see following command output, it says "Number of available ports: 0", so that shows port number is updated after destroy event, isn't it? >> testpmd> show port summary all >> Number of available ports: 0 >> Port MAC Address Name Driver Status Link >> testpmd> >> `` >> >> >> . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-06-06 16:12 ` Ferruh Yigit @ 2023-06-07 10:11 ` lihuisong (C) 2023-06-15 2:21 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-06-07 10:11 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen, lihuisong 在 2023/6/7 0:12, Ferruh Yigit 写道: > On 2/28/2023 2:21 AM, lihuisong (C) wrote: >> 在 2023/2/16 0:09, Ferruh Yigit 写道: >>> On 1/12/2023 2:44 AM, lihuisong (C) wrote: >>>> 在 2023/1/11 20:51, Ferruh Yigit 写道: >>>>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>>>> The driver assignment was moved back at the end of the device probing >>>>>> because there is no something to use rte_driver during the phase of >>>>>> probing. See commit 391797f04208 ("drivers/bus: move driver assignment >>>>>> to end of probing") >>>>>> >>>>>> However, it is necessary for probing callback to reference rte_driver >>>>>> before probing. For example, probing callback may call some APIs which >>>>>> access the rte_pci_driver::driver by the device::driver pointer to get >>>>>> driver information. In this case, a segment fault will occur in probing >>>>>> callback if there is not this assignment. >>>>>> >>>>> Probing callback gets driver as parameter, so callback function can >>>>> access it via 'drv->driver', is there a specific usecase that >>>>> 'dev->device->driver' needs to be accessed explicitly? >>>>> >>>>> I assume this is related to coming patches that setting up device in >>>>> testpmd event callback, but can you please clarify exact need. >>>> For example, rte_eth_dev_info_get is called in this event callback to get >>>> driver name. >>>> >> Hi Ferruh, >> >> Sorry for the delay. I missed this email. >> Thanks for your review. >>> Why 'rte_eth_dev_info_get()' is called in the event called at first place? >> There is no limitation on rte_eth_dev_info_get() in event callback. >> The upper layer is entirely possible to use. >> After all, it is more convenient for app to use this pointer to get >> driver information in a probing callback. >> > I think there is a logical limit in calling 'rte_eth_dev_info_get()' in > NEW event callback, we may document this if it is missing. > > The NEW event callback is to let application do proper tasks before port > addition finalized in the ethdev layer, but you are trying to call > 'rte_eth_dev_info_get()' in the callback when it is not ready yet. We probably shouldn't add this logical limit. In terms of application perception, app may need to do something when a new or destory event is received, instead of just for printing a message. > >>> This set updates multiple things to extend 'RTE_ETH_EVENT_NEW' event >>> callback function support, like: >>> >>> - Assign device driver *before* probing completed, so that even callback >>> can run 'rte_eth_dev_info_get()' >>> >>> - Add a new ethdev state so that port can be recognized as valid port in >>> the even callback. >> Yes >>> - Stop forwarding implicitly in even callback in case event callback run >>> while forwarding is on. >> But this is a modification for tesptmd to make it work well. >>> All looks to me hack/complexity to make a specific case work, which is >>> make secondary *testmp* application work with attached/detached device. >> It is not just a testpmd application case, but, I think, still exists in >> actual case. >> Because eal lib supports hotplugging device on primary and secondary >> process and the communication each other when attach or detach device. >> The reason why no one has ever put forward this question, I think it may >> be attributed to the fact that the scene is not or rarely tested. >>> And finally patch 4/5 adds port setup to testpmd event callback for this. >>> >>> >>> I understand the intention, but I disagree with bus and ethdev level >>> changes for this. >> I'm just raising this issue, and we can discuss how to deal with it >> together.😁 >>> Event callback may not be only way to share port attach/detach >>> information between primary and secondary, there is a MP socket and >>> 'rte_mp_handle' thread to handle communication between primary and >>> secondary process, this should be able to use carrying device >>> information, as far as I remember this is why it is introduced at first >>> place. >>> >>> Did you consider using MP socket for your use case? >> Actually, the probing event callback called in patch 4/5 is the result >> of the MP socket communication (please see hogplug_mp.c). >>> Following is a sample usage: >>> >>> Primary: >>> started as: >>> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 0-1 >>> --log-level=*:debug -- -i --num-procs=2 --proc-id=0 >>> >>> `` >>> testpmd> show port summary all >>> Number of available ports: 0 >>> Port MAC Address Name Driver Status Link >>> >>> testpmd> port attach net_null0 >>> Attaching a new port... >>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>> vdev_probe_all_drivers(): Search driver to probe device net_null0 >>> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >>> rte_pmd_null_probe(): Configure pmd_null: packet size is 64, packet copy >>> is disabled >>> eth_dev_null_create(): Creating null ethdev on numa socket 0 >>> EAL: request: eal_dev_mp_request >>> EAL: msg: bus_vdev_mp >>> vdev_action(): send vdev, net_null0 >>> EAL: sendmsg: bus_vdev_mp >>> EAL: reply: bus_vdev_mp >>> EAL: msg: eal_dev_mp_request >>> Port 0 is attached. Now total ports is 1 >>> Done >>> >>> testpmd> show port summary all >>> Number of available ports: 1 >>> Port MAC Address Name Driver Status Link >>> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >>> >>> testpmd> port detach 0 >>> Port was not closed >>> Removing a device... >>> EAL: request: eal_dev_mp_request >>> EAL: msg: eal_dev_mp_request >>> eth_dev_close(): Closing null ethdev on NUMA socket 0 >>> Port 0 is closed >>> Device is detached >>> Now total ports is 0 >> Please note the log *"Now total ports is 0"*, >> which indicates the port number is updated if we detached device in this >> process. >> > Yes, as expected. Yeah, it is just expected for the process did attatch or dettach operation. Because the thread for updating the port number info (in setup_attached_port) and doing probe thread is the same. please see attach_port(). This is ok for testpmd that does not support multiple processes. But testpmd support multiple processes now. > > >>> Done >>> >>> testpmd> show port summary all >>> Number of available ports: 0 >>> Port MAC Address Name Driver Status Link >>> testpmd> >>> >>> `` >>> >>> Secondary: >>> started as: >>> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 2-3 >>> --log-level=*:debug -- -i --num-procs=2 --proc-id=1 >>> >>> `` >>> testpmd> show port summary all >>> Number of available ports: 0 >>> Port MAC Address Name Driver Status Link >>> >>> testpmd> EAL: msg: eal_dev_mp_request >>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>> EAL: request: bus_vdev_mp >>> EAL: msg: bus_vdev_mp >>> vdev_action(): receive vdev, net_null0 >>> EAL: msg: bus_vdev_mp >>> vdev_scan(): Received 1 vdevs >>> vdev_probe_all_drivers(): Search driver to probe device net_null0 >>> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >>> EAL: reply: eal_dev_mp_request >>> >>> testpmd> show port summary all >>> Number of available ports: 1 >>> Port MAC Address Name Driver Status Link >>> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >>> >>> testpmd> EAL: msg: eal_dev_mp_request >>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>> eth_dev_close(): Closing null ethdev on NUMA socket 4294967295 >>> Port 0 is closed >>> EAL: reply: eal_dev_mp_request >> But the port number in this process does not be updated after finishing >> the destroy event. >> That's the problem. >> > Please see following command output, > it says "Number of available ports: 0", Here comes from rte_eth_dev_count_avail(). > so that shows port number is updated after destroy event, isn't it? no. There are three global variables, "fwd_ports_ids[]", "ports[]", "nb_ports" maintained in testpmd. The secondary doesn't update these info after receiving a destory or new message in your test. > > >>> testpmd> show port summary all >>> Number of available ports: 0 >>> Port MAC Address Name Driver Status Link >>> testpmd> >>> `` >>> >>> >>> . > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing 2023-06-07 10:11 ` lihuisong (C) @ 2023-06-15 2:21 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-06-15 2:21 UTC (permalink / raw) To: Ferruh Yigit, dev, thomas Cc: liudongdong3, fengchengwen, andrew.rybchenko, liuyonglong Hi Ferruh, add the call stack as belows. 在 2023/6/7 18:11, lihuisong (C) 写道: > > 在 2023/6/7 0:12, Ferruh Yigit 写道: >> On 2/28/2023 2:21 AM, lihuisong (C) wrote: >>> 在 2023/2/16 0:09, Ferruh Yigit 写道: >>>> On 1/12/2023 2:44 AM, lihuisong (C) wrote: >>>>> 在 2023/1/11 20:51, Ferruh Yigit 写道: >>>>>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>>>>> The driver assignment was moved back at the end of the device >>>>>>> probing >>>>>>> because there is no something to use rte_driver during the phase of >>>>>>> probing. See commit 391797f04208 ("drivers/bus: move driver >>>>>>> assignment >>>>>>> to end of probing") >>>>>>> >>>>>>> However, it is necessary for probing callback to reference >>>>>>> rte_driver >>>>>>> before probing. For example, probing callback may call some APIs >>>>>>> which >>>>>>> access the rte_pci_driver::driver by the device::driver pointer >>>>>>> to get >>>>>>> driver information. In this case, a segment fault will occur in >>>>>>> probing >>>>>>> callback if there is not this assignment. >>>>>>> >>>>>> Probing callback gets driver as parameter, so callback function can >>>>>> access it via 'drv->driver', is there a specific usecase that >>>>>> 'dev->device->driver' needs to be accessed explicitly? >>>>>> >>>>>> I assume this is related to coming patches that setting up device in >>>>>> testpmd event callback, but can you please clarify exact need. >>>>> For example, rte_eth_dev_info_get is called in this event callback >>>>> to get >>>>> driver name. >>>>> >>> Hi Ferruh, >>> >>> Sorry for the delay. I missed this email. >>> Thanks for your review. >>>> Why 'rte_eth_dev_info_get()' is called in the event called at first >>>> place? >>> There is no limitation on rte_eth_dev_info_get() in event callback. >>> The upper layer is entirely possible to use. >>> After all, it is more convenient for app to use this pointer to get >>> driver information in a probing callback. >>> >> I think there is a logical limit in calling 'rte_eth_dev_info_get()' in >> NEW event callback, we may document this if it is missing. >> >> The NEW event callback is to let application do proper tasks before port >> addition finalized in the ethdev layer, but you are trying to call >> 'rte_eth_dev_info_get()' in the callback when it is not ready yet. > We probably shouldn't add this logical limit. > In terms of application perception, app may need to do something when > a new or destory event is received, instead of just for printing a > message. In addition, if no this patch(1/5), a segment fault will occur when attach a new device. The call stack is as belows: --> testpmd> port attach 0000:7d:00.0 Attaching a new port... EAL: Using IOMMU type 1 (Type 1) EAL: Ignore mapping IO port bar(1) EAL: Ignore mapping IO port bar(3) EAL: Probe PCI driver: net_hns3 (19e5:a222) device: 0000:7d:00.0 (socket 0) Thread 1 "dpdk-testpmd" received signal SIGSEGV, Segmentation fault. rte_eth_dev_info_get (port_id=0, dev_info=0x17f95ba80) at ../lib/ethdev/rte_ethdev.c:3708 3708 dev_info->driver_name = dev->device->driver->name; Missing separate debuginfos, use: dnf debuginfo-install libatomic-10.3.1-10.oe2203.aarch64 libnl3-3.5.0-4.oe2203.aarch64 openssl-libs-1.1.1m-1.oe2203.aarch64 zlib-1.2.11-19.oe2203.aarch64 (gdb) bt #0 rte_eth_dev_info_get (port_id=0, dev_info=0x17f95ba80) at ../lib/ethdev/rte_ethdev.c:3708 #1 0x00000000005e6038 in eth_dev_info_get_print_err (port_id=0, dev_info=0x17f95ba80) at ../app/test-pmd/util.c:441 #2 0x00000000005d5654 in init_config_port_offloads (pid=0, socket_id=0) at ../app/test-pmd/testpmd.c:1632 #3 0x00000000005d5ec0 in reconfig (new_port_id=0, socket_id=0) at ../app/test-pmd/testpmd.c:1828 #4 0x00000000005dafa4 in setup_attached_port (pi=0) at ../app/test-pmd/testpmd.c:3639 #5 0x00000000005dbd14 in eth_event_callback (port_id=0, type=RTE_ETH_EVENT_NEW, param=0x0, ret_param=0x0) at ../app/test-pmd/testpmd.c:3959 #6 0x00000000008677dc in rte_eth_dev_callback_process (dev=0x1826fd80 <rte_eth_devices>, event=RTE_ETH_EVENT_NEW, ret_param=0x0) at ../lib/ethdev/ethdev_driver.c:188 #7 0x0000000000867898 in rte_eth_dev_probing_finish (dev=0x1826fd80 <rte_eth_devices>) at ../lib/ethdev/ethdev_driver.c:212 #8 0x00000000055621c8 in rte_eth_dev_pci_generic_probe (pci_dev=0x18500470, private_data_size=8448, dev_init=0x5571454 <hns3_dev_init>) at ../lib/ethdev/ethdev_pci.h:139 #9 0x00000000055717bc in eth_hns3_pci_probe (pci_drv=0x17e46728 <rte_hns3_pmd>, pci_dev=0x18500470) at ../drivers/net/hns3/hns3_ethdev.c:6577 #10 0x0000000000970474 in rte_pci_probe_one_driver (dr=0x17e46728 <rte_hns3_pmd>, dev=0x18500470) at ../drivers/bus/pci/pci_common.c:312 #11 0x00000000009706e0 in pci_probe_all_drivers (dev=0x18500470) at ../drivers/bus/pci/pci_common.c:396 #12 0x0000000000970f78 in pci_plug (dev=0x18500480) at ../drivers/bus/pci/pci_common.c:670 #13 0x00000000008e31d4 in local_dev_probe (devargs=0xffffffffb6f0 "0000:7d:00.0", new_dev=0xffffffff9478) at ../lib/eal/common/eal_common_dev.c:212 #14 0x00000000008e3330 in rte_dev_probe (devargs=0xffffffffb6f0 "0000:7d:00.0") at ../lib/eal/common/eal_common_dev.c:264 #15 0x00000000005daeb4 in attach_port (identifier=0xffffffffb6f0 "0000:7d:00.0") at ../app/test-pmd/testpmd.c:3608 #16 0x0000000000579a74 in cmd_operate_attach_port_parsed (parsed_result=0xffffffffb5f0, cl=0x184f9010, data=0x0) at ../app/test-pmd/cmdline.c:1194 --Type <RET> for more, q to quit, c to continue without paging-- #17 0x00000000008624a4 in __cmdline_parse (cl=0x184f9010, buf=0x184f9058 "port attach 0000:7d:00.0\n", call_fn=true) at ../lib/cmdline/cmdline_parse.c:294 #18 0x00000000008624dc in cmdline_parse (cl=0x184f9010, buf=0x184f9058 "port attach 0000:7d:00.0\n") at ../lib/cmdline/cmdline_parse.c:302 #19 0x0000000000860308 in cmdline_valid_buffer (rdl=0x184f9020, buf=0x184f9058 "port attach 0000:7d:00.0\n", size=26) at ../lib/cmdline/cmdline.c:24 #20 0x000000000086588c in rdline_char_in (rdl=0x184f9020, c=10 '\n') at ../lib/cmdline/cmdline_rdline.c:444 #21 0x0000000000860750 in cmdline_in (cl=0x184f9010, buf=0xfffffffff77f "\n\220\367\377\377\377\377", size=1) at ../lib/cmdline/cmdline.c:146 #22 0x0000000000860a14 in cmdline_interact (cl=0x184f9010) at ../lib/cmdline/cmdline.c:226 #23 0x0000000000587908 in prompt () at ../app/test-pmd/cmdline.c:13065 #24 0x00000000005ddaf4 in main (argc=6, argv=0xfffffffffad8) at ../app/test-pmd/testpmd.c:4690 >> >>>> This set updates multiple things to extend 'RTE_ETH_EVENT_NEW' event >>>> callback function support, like: >>>> >>>> - Assign device driver *before* probing completed, so that even >>>> callback >>>> can run 'rte_eth_dev_info_get()' >>>> >>>> - Add a new ethdev state so that port can be recognized as valid >>>> port in >>>> the even callback. >>> Yes >>>> - Stop forwarding implicitly in even callback in case event >>>> callback run >>>> while forwarding is on. >>> But this is a modification for tesptmd to make it work well. >>>> All looks to me hack/complexity to make a specific case work, which is >>>> make secondary *testmp* application work with attached/detached >>>> device. >>> It is not just a testpmd application case, but, I think, still >>> exists in >>> actual case. >>> Because eal lib supports hotplugging device on primary and secondary >>> process and the communication each other when attach or detach device. >>> The reason why no one has ever put forward this question, I think it >>> may >>> be attributed to the fact that the scene is not or rarely tested. >>>> And finally patch 4/5 adds port setup to testpmd event callback for >>>> this. >>>> >>>> >>>> I understand the intention, but I disagree with bus and ethdev level >>>> changes for this. >>> I'm just raising this issue, and we can discuss how to deal with it >>> together.😁 >>>> Event callback may not be only way to share port attach/detach >>>> information between primary and secondary, there is a MP socket and >>>> 'rte_mp_handle' thread to handle communication between primary and >>>> secondary process, this should be able to use carrying device >>>> information, as far as I remember this is why it is introduced at >>>> first >>>> place. >>>> >>>> Did you consider using MP socket for your use case? >>> Actually, the probing event callback called in patch 4/5 is the result >>> of the MP socket communication (please see hogplug_mp.c). >>>> Following is a sample usage: >>>> >>>> Primary: >>>> started as: >>>> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 0-1 >>>> --log-level=*:debug -- -i --num-procs=2 --proc-id=0 >>>> >>>> `` >>>> testpmd> show port summary all >>>> Number of available ports: 0 >>>> Port MAC Address Name Driver Status Link >>>> >>>> testpmd> port attach net_null0 >>>> Attaching a new port... >>>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>>> vdev_probe_all_drivers(): Search driver to probe device net_null0 >>>> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >>>> rte_pmd_null_probe(): Configure pmd_null: packet size is 64, packet >>>> copy >>>> is disabled >>>> eth_dev_null_create(): Creating null ethdev on numa socket 0 >>>> EAL: request: eal_dev_mp_request >>>> EAL: msg: bus_vdev_mp >>>> vdev_action(): send vdev, net_null0 >>>> EAL: sendmsg: bus_vdev_mp >>>> EAL: reply: bus_vdev_mp >>>> EAL: msg: eal_dev_mp_request >>>> Port 0 is attached. Now total ports is 1 >>>> Done >>>> >>>> testpmd> show port summary all >>>> Number of available ports: 1 >>>> Port MAC Address Name Driver Status Link >>>> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >>>> >>>> testpmd> port detach 0 >>>> Port was not closed >>>> Removing a device... >>>> EAL: request: eal_dev_mp_request >>>> EAL: msg: eal_dev_mp_request >>>> eth_dev_close(): Closing null ethdev on NUMA socket 0 >>>> Port 0 is closed >>>> Device is detached >>>> Now total ports is 0 >>> Please note the log *"Now total ports is 0"*, >>> which indicates the port number is updated if we detached device in >>> this >>> process. >>> >> Yes, as expected. > Yeah, it is just expected for the process did attatch or dettach > operation. > Because the thread for updating the port number info (in > setup_attached_port) and doing probe thread is the same. > please see attach_port(). > This is ok for testpmd that does not support multiple processes. But > testpmd support multiple processes now. >> >> >>>> Done >>>> >>>> testpmd> show port summary all >>>> Number of available ports: 0 >>>> Port MAC Address Name Driver Status Link >>>> testpmd> >>>> >>>> `` >>>> >>>> Secondary: >>>> started as: >>>> sudo ./build/app/dpdk-testpmd --no-pci --proc-type=auto -l 2-3 >>>> --log-level=*:debug -- -i --num-procs=2 --proc-id=1 >>>> >>>> `` >>>> testpmd> show port summary all >>>> Number of available ports: 0 >>>> Port MAC Address Name Driver Status Link >>>> >>>> testpmd> EAL: msg: eal_dev_mp_request >>>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>>> EAL: request: bus_vdev_mp >>>> EAL: msg: bus_vdev_mp >>>> vdev_action(): receive vdev, net_null0 >>>> EAL: msg: bus_vdev_mp >>>> vdev_scan(): Received 1 vdevs >>>> vdev_probe_all_drivers(): Search driver to probe device net_null0 >>>> rte_pmd_null_probe(): Initializing pmd_null for net_null0 >>>> EAL: reply: eal_dev_mp_request >>>> >>>> testpmd> show port summary all >>>> Number of available ports: 1 >>>> Port MAC Address Name Driver Status Link >>>> 0 DE:E5:79:00:A9:68 net_null0 net_null down 10 Gbps >>>> >>>> testpmd> EAL: msg: eal_dev_mp_request >>>> dpaa: rte_dpaa_bus_parse(): Parse device name (net_null0 ) >>>> fslmc: rte_fslmc_parse(): Parsing dev=(net_null0 ) >>>> fslmc: rte_fslmc_parse(): Unknown or unsupported device (net_null0 ) >>>> eth_dev_close(): Closing null ethdev on NUMA socket 4294967295 >>>> Port 0 is closed >>>> EAL: reply: eal_dev_mp_request >>> But the port number in this process does not be updated after finishing >>> the destroy event. >>> That's the problem. >>> >> Please see following command output, >> it says "Number of available ports: 0", > Here comes from rte_eth_dev_count_avail(). >> so that shows port number is updated after destroy event, isn't it? > no. > There are three global variables, "fwd_ports_ids[]", "ports[]", > "nb_ports" maintained in testpmd. > The secondary doesn't update these info after receiving a destory or > new message in your test. >> >> >>>> testpmd> show port summary all >>>> Number of available ports: 0 >>>> Port MAC Address Name Driver Status Link >>>> testpmd> >>>> `` >>>> >>>> >>>> . >> . > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 2/5] ethdev: fix skip valid port in probing callback 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 9:26 ` [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2022-12-06 9:26 ` [PATCH V4 3/5] app/testpmd: check the validity of the port Huisong Li ` (4 subsequent siblings) 6 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is RTE_ETH_DEV_UNUSED, this port will be skipped. Currently, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and to RTE_ETH_DEV_ATTACHED when definitely probed. And this port is valid if its device state is ALLOCATED or ATTACHED. Naturally, a lot of places had to be modified for this change. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index b3de490d36..7b08353ed5 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6003,7 +6003,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_in_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e55be8720e..949d8c1367 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3014,7 +3014,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_in_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..7868958dff 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_in_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_in_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_in_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 6a550cfc83..e04f88fe46 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1542,6 +1542,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is in used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_in_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 94b8fba5d7..362e5c2dec 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_in_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index 838b3a8f9f..a78d503e59 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_in_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 5d5e18db1e..a225141937 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -325,7 +325,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -372,7 +372,7 @@ int rte_eth_dev_is_valid_port(uint16_t port_id) { if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) return 0; else return 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index c129ca1eaf..459ad44021 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2000,7 +2000,9 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 17201fbe0f..66d70f1aac 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -327,4 +327,5 @@ INTERNAL { rte_eth_representor_id_get; rte_eth_switch_domain_alloc; rte_eth_switch_domain_free; + rte_eth_dev_in_used; }; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 2/5] ethdev: fix skip valid port in probing callback 2022-12-06 9:26 ` [PATCH V4 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 4:12 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 12:51 UTC (permalink / raw) To: Huisong Li, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 12/6/2022 9:26 AM, Huisong Li wrote: > The event callback in application may use the macro RTE_ETH_FOREACH_DEV to > iterate over all enabled ports to do something(like, verifying the port id > validity) when receive a probing event. If the ethdev state of a port is > RTE_ETH_DEV_UNUSED, this port will be skipped. > > Currently, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing > event. It means that probing callback will skip this port. But this > assignment can not move to front of probing notification. See > commit be8cd210379a ("ethdev: fix port probing notification") > OK to abstract ethdev state check to a function, 'rte_eth_dev_in_used'. BTW, I guess intention is to say 'is' instead of 'in', like 'rte_eth_dev_is_used'. But not sure if new state is necessary. I can see from next patches that if testpmd event callback wants to validate port_id, it is using 'port_id_is_invalid()' function which is using 'RTE_ETH_FOREACH_DEV' macro. And using 'RTE_ETH_FOREACH_DEV' macro in callback function has the problem you described above. 1) Do we really need to validate port_id in the event callback, callback provided 'port_id' from 'rte_eth_dev_callback_process()' API, from 'dev->data->port_id'. I am not sure if there is case that 'dev->data->port_id' can be invalid. 2) is there any other specific usecase to use 'RTE_ETH_FOREACH_DEV' in the 'RTE_ETH_EVENT_NEW' event callback function? > So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev > state to RTE_ETH_DEV_ALLOCATED before pushing probing event and to > RTE_ETH_DEV_ATTACHED when definitely probed. And this port is valid if its > device state is ALLOCATED or ATTACHED. Naturally, a lot of places had to be > modified for this change. > > Fixes: be8cd210379a ("ethdev: fix port probing notification") > Cc: stable@dpdk.org > > Signed-off-by: Huisong Li <lihuisong@huawei.com> > --- > drivers/net/bnxt/bnxt_ethdev.c | 3 ++- > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 ++++++++++--- > lib/ethdev/ethdev_driver.h | 12 ++++++++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 ++-- > lib/ethdev/rte_ethdev.h | 4 +++- > lib/ethdev/version.map | 1 + > 9 files changed, 33 insertions(+), 10 deletions(-) > > diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c > index b3de490d36..7b08353ed5 100644 > --- a/drivers/net/bnxt/bnxt_ethdev.c > +++ b/drivers/net/bnxt/bnxt_ethdev.c > @@ -6003,7 +6003,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) > > PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); > > - if (eth_dev->state != RTE_ETH_DEV_UNUSED) > + > + if (rte_eth_dev_in_used(eth_dev->state)) > bnxt_dev_close_op(eth_dev); > > return 0; > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c > index e55be8720e..949d8c1367 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -3014,7 +3014,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) > while (port_id < RTE_MAX_ETHPORTS) { > struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > - if (dev->state != RTE_ETH_DEV_UNUSED && > + if (rte_eth_dev_in_used(dev->state) && > dev->device && > (dev->device == odev || > (dev->device->driver && > diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c > index 0be1e8ca04..7868958dff 100644 > --- a/lib/ethdev/ethdev_driver.c > +++ b/lib/ethdev/ethdev_driver.c > @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) > for (i = 0; i < RTE_MAX_ETHPORTS; i++) { > /* Using shared name field to find a free port. */ > if (eth_dev_shared_data->data[i].name[0] == '\0') { > - RTE_ASSERT(rte_eth_devices[i].state == > - RTE_ETH_DEV_UNUSED); > + RTE_ASSERT(!rte_eth_dev_in_used( > + rte_eth_devices[i].state)); > return i; > } > } > @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) > if (rte_eal_process_type() == RTE_PROC_SECONDARY) > eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); > > + dev->state = RTE_ETH_DEV_ALLOCATED; > rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); > > dev->state = RTE_ETH_DEV_ATTACHED; > } > > +bool rte_eth_dev_in_used(uint16_t dev_state) > +{ > + return dev_state == RTE_ETH_DEV_ALLOCATED || > + dev_state == RTE_ETH_DEV_ATTACHED; > +} > + > int > rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) > { > @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) > > eth_dev_shared_data_prepare(); > > - if (eth_dev->state != RTE_ETH_DEV_UNUSED) > + if (rte_eth_dev_in_used(eth_dev->state)) > rte_eth_dev_callback_process(eth_dev, > RTE_ETH_EVENT_DESTROY, NULL); > > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > index 6a550cfc83..e04f88fe46 100644 > --- a/lib/ethdev/ethdev_driver.h > +++ b/lib/ethdev/ethdev_driver.h > @@ -1542,6 +1542,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, > __rte_internal > void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); > > +/** > + * Check if a Ethernet device state is in used or not > + * > + * @param dev_state > + * The state of the Ethernet device > + * @return > + * - true if the state of the Ethernet device is allocated or attached > + * - false if this state is neither allocated nor attached > + */ > +__rte_internal > +bool rte_eth_dev_in_used(uint16_t dev_state); > + > /** > * Create memzone for HW rings. > * malloc can't be used as the physical address is needed. > diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h > index 94b8fba5d7..362e5c2dec 100644 > --- a/lib/ethdev/ethdev_pci.h > +++ b/lib/ethdev/ethdev_pci.h > @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, > * eth device has been released. > */ > if (rte_eal_process_type() == RTE_PROC_SECONDARY && > - eth_dev->state == RTE_ETH_DEV_UNUSED) > + !rte_eth_dev_in_used(eth_dev->state)) > return 0; > > if (dev_uninit) { > diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c > index 838b3a8f9f..a78d503e59 100644 > --- a/lib/ethdev/rte_class_eth.c > +++ b/lib/ethdev/rte_class_eth.c > @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, > const struct rte_kvargs *kvlist = arg->kvlist; > unsigned int pair; > > - if (edev->state == RTE_ETH_DEV_UNUSED) > + if (!rte_eth_dev_in_used(edev->state)) > return -1; > if (arg->device != NULL && arg->device != edev->device) > return -1; > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c > index 5d5e18db1e..a225141937 100644 > --- a/lib/ethdev/rte_ethdev.c > +++ b/lib/ethdev/rte_ethdev.c > @@ -325,7 +325,7 @@ uint16_t > rte_eth_find_next(uint16_t port_id) > { > while (port_id < RTE_MAX_ETHPORTS && > - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) > + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) > port_id++; > > if (port_id >= RTE_MAX_ETHPORTS) > @@ -372,7 +372,7 @@ int > rte_eth_dev_is_valid_port(uint16_t port_id) > { > if (port_id >= RTE_MAX_ETHPORTS || > - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) > + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) > return 0; > else > return 1; > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index c129ca1eaf..459ad44021 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -2000,7 +2000,9 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, > enum rte_eth_dev_state { > /** Device is unused before being probed. */ > RTE_ETH_DEV_UNUSED = 0, > - /** Device is attached when allocated in probing. */ > + /** Device is allocated and is set before reporting new event. */ > + RTE_ETH_DEV_ALLOCATED, > + /** Device is attached when definitely probed. */ > RTE_ETH_DEV_ATTACHED, > /** Device is in removed state when plug-out is detected. */ > RTE_ETH_DEV_REMOVED, > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map > index 17201fbe0f..66d70f1aac 100644 > --- a/lib/ethdev/version.map > +++ b/lib/ethdev/version.map > @@ -327,4 +327,5 @@ INTERNAL { > rte_eth_representor_id_get; > rte_eth_switch_domain_alloc; > rte_eth_switch_domain_free; > + rte_eth_dev_in_used; > }; ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 2/5] ethdev: fix skip valid port in probing callback 2023-01-11 12:51 ` Ferruh Yigit @ 2023-01-12 4:12 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-12 4:12 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen 在 2023/1/11 20:51, Ferruh Yigit 写道: > On 12/6/2022 9:26 AM, Huisong Li wrote: >> The event callback in application may use the macro RTE_ETH_FOREACH_DEV to >> iterate over all enabled ports to do something(like, verifying the port id >> validity) when receive a probing event. If the ethdev state of a port is >> RTE_ETH_DEV_UNUSED, this port will be skipped. >> >> Currently, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing >> event. It means that probing callback will skip this port. But this >> assignment can not move to front of probing notification. See >> commit be8cd210379a ("ethdev: fix port probing notification") >> > OK to abstract ethdev state check to a function, 'rte_eth_dev_in_used'. > BTW, I guess intention is to say 'is' instead of 'in', like > 'rte_eth_dev_is_used'. Ack > > But not sure if new state is necessary. > I can see from next patches that if testpmd event callback wants to > validate port_id, it is using 'port_id_is_invalid()' function which is > using 'RTE_ETH_FOREACH_DEV' macro. > And using 'RTE_ETH_FOREACH_DEV' macro in callback function has the > problem you described above. > > 1) Do we really need to validate port_id in the event callback, callback > provided 'port_id' from 'rte_eth_dev_callback_process()' API, from > 'dev->data->port_id'. I am not sure if there is case that > 'dev->data->port_id' can be invalid. > > 2) is there any other specific usecase to use 'RTE_ETH_FOREACH_DEV' in > the 'RTE_ETH_EVENT_NEW' event callback function? Reply 1) and 2): this check should be useful for failsafe. > > > >> So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev >> state to RTE_ETH_DEV_ALLOCATED before pushing probing event and to >> RTE_ETH_DEV_ATTACHED when definitely probed. And this port is valid if its >> device state is ALLOCATED or ATTACHED. Naturally, a lot of places had to be >> modified for this change. >> >> Fixes: be8cd210379a ("ethdev: fix port probing notification") >> Cc: stable@dpdk.org >> >> Signed-off-by: Huisong Li <lihuisong@huawei.com> >> --- >> drivers/net/bnxt/bnxt_ethdev.c | 3 ++- >> drivers/net/mlx5/mlx5.c | 2 +- >> lib/ethdev/ethdev_driver.c | 13 ++++++++++--- >> lib/ethdev/ethdev_driver.h | 12 ++++++++++++ >> lib/ethdev/ethdev_pci.h | 2 +- >> lib/ethdev/rte_class_eth.c | 2 +- >> lib/ethdev/rte_ethdev.c | 4 ++-- >> lib/ethdev/rte_ethdev.h | 4 +++- >> lib/ethdev/version.map | 1 + >> 9 files changed, 33 insertions(+), 10 deletions(-) >> >> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c >> index b3de490d36..7b08353ed5 100644 >> --- a/drivers/net/bnxt/bnxt_ethdev.c >> +++ b/drivers/net/bnxt/bnxt_ethdev.c >> @@ -6003,7 +6003,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) >> >> PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); >> >> - if (eth_dev->state != RTE_ETH_DEV_UNUSED) >> + >> + if (rte_eth_dev_in_used(eth_dev->state)) >> bnxt_dev_close_op(eth_dev); >> >> return 0; >> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c >> index e55be8720e..949d8c1367 100644 >> --- a/drivers/net/mlx5/mlx5.c >> +++ b/drivers/net/mlx5/mlx5.c >> @@ -3014,7 +3014,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) >> while (port_id < RTE_MAX_ETHPORTS) { >> struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> >> - if (dev->state != RTE_ETH_DEV_UNUSED && >> + if (rte_eth_dev_in_used(dev->state) && >> dev->device && >> (dev->device == odev || >> (dev->device->driver && >> diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c >> index 0be1e8ca04..7868958dff 100644 >> --- a/lib/ethdev/ethdev_driver.c >> +++ b/lib/ethdev/ethdev_driver.c >> @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) >> for (i = 0; i < RTE_MAX_ETHPORTS; i++) { >> /* Using shared name field to find a free port. */ >> if (eth_dev_shared_data->data[i].name[0] == '\0') { >> - RTE_ASSERT(rte_eth_devices[i].state == >> - RTE_ETH_DEV_UNUSED); >> + RTE_ASSERT(!rte_eth_dev_in_used( >> + rte_eth_devices[i].state)); >> return i; >> } >> } >> @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) >> if (rte_eal_process_type() == RTE_PROC_SECONDARY) >> eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); >> >> + dev->state = RTE_ETH_DEV_ALLOCATED; >> rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); >> >> dev->state = RTE_ETH_DEV_ATTACHED; >> } >> >> +bool rte_eth_dev_in_used(uint16_t dev_state) >> +{ >> + return dev_state == RTE_ETH_DEV_ALLOCATED || >> + dev_state == RTE_ETH_DEV_ATTACHED; >> +} >> + >> int >> rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) >> { >> @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) >> >> eth_dev_shared_data_prepare(); >> >> - if (eth_dev->state != RTE_ETH_DEV_UNUSED) >> + if (rte_eth_dev_in_used(eth_dev->state)) >> rte_eth_dev_callback_process(eth_dev, >> RTE_ETH_EVENT_DESTROY, NULL); >> >> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h >> index 6a550cfc83..e04f88fe46 100644 >> --- a/lib/ethdev/ethdev_driver.h >> +++ b/lib/ethdev/ethdev_driver.h >> @@ -1542,6 +1542,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, >> __rte_internal >> void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); >> >> +/** >> + * Check if a Ethernet device state is in used or not >> + * >> + * @param dev_state >> + * The state of the Ethernet device >> + * @return >> + * - true if the state of the Ethernet device is allocated or attached >> + * - false if this state is neither allocated nor attached >> + */ >> +__rte_internal >> +bool rte_eth_dev_in_used(uint16_t dev_state); >> + >> /** >> * Create memzone for HW rings. >> * malloc can't be used as the physical address is needed. >> diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h >> index 94b8fba5d7..362e5c2dec 100644 >> --- a/lib/ethdev/ethdev_pci.h >> +++ b/lib/ethdev/ethdev_pci.h >> @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, >> * eth device has been released. >> */ >> if (rte_eal_process_type() == RTE_PROC_SECONDARY && >> - eth_dev->state == RTE_ETH_DEV_UNUSED) >> + !rte_eth_dev_in_used(eth_dev->state)) >> return 0; >> >> if (dev_uninit) { >> diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c >> index 838b3a8f9f..a78d503e59 100644 >> --- a/lib/ethdev/rte_class_eth.c >> +++ b/lib/ethdev/rte_class_eth.c >> @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, >> const struct rte_kvargs *kvlist = arg->kvlist; >> unsigned int pair; >> >> - if (edev->state == RTE_ETH_DEV_UNUSED) >> + if (!rte_eth_dev_in_used(edev->state)) >> return -1; >> if (arg->device != NULL && arg->device != edev->device) >> return -1; >> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c >> index 5d5e18db1e..a225141937 100644 >> --- a/lib/ethdev/rte_ethdev.c >> +++ b/lib/ethdev/rte_ethdev.c >> @@ -325,7 +325,7 @@ uint16_t >> rte_eth_find_next(uint16_t port_id) >> { >> while (port_id < RTE_MAX_ETHPORTS && >> - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) >> + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) >> port_id++; >> >> if (port_id >= RTE_MAX_ETHPORTS) >> @@ -372,7 +372,7 @@ int >> rte_eth_dev_is_valid_port(uint16_t port_id) >> { >> if (port_id >= RTE_MAX_ETHPORTS || >> - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) >> + !rte_eth_dev_in_used(rte_eth_devices[port_id].state)) >> return 0; >> else >> return 1; >> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h >> index c129ca1eaf..459ad44021 100644 >> --- a/lib/ethdev/rte_ethdev.h >> +++ b/lib/ethdev/rte_ethdev.h >> @@ -2000,7 +2000,9 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, >> enum rte_eth_dev_state { >> /** Device is unused before being probed. */ >> RTE_ETH_DEV_UNUSED = 0, >> - /** Device is attached when allocated in probing. */ >> + /** Device is allocated and is set before reporting new event. */ >> + RTE_ETH_DEV_ALLOCATED, >> + /** Device is attached when definitely probed. */ >> RTE_ETH_DEV_ATTACHED, >> /** Device is in removed state when plug-out is detected. */ >> RTE_ETH_DEV_REMOVED, >> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map >> index 17201fbe0f..66d70f1aac 100644 >> --- a/lib/ethdev/version.map >> +++ b/lib/ethdev/version.map >> @@ -327,4 +327,5 @@ INTERNAL { >> rte_eth_representor_id_get; >> rte_eth_switch_domain_alloc; >> rte_eth_switch_domain_free; >> + rte_eth_dev_in_used; >> }; > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 3/5] app/testpmd: check the validity of the port 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 9:26 ` [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2022-12-06 9:26 ` [PATCH V4 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2022-12-06 9:26 ` [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (3 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 134d79a555..bc25703490 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3798,14 +3798,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (2 preceding siblings ...) 2022-12-06 9:26 ` [PATCH V4 3/5] app/testpmd: check the validity of the port Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2022-12-06 9:26 ` [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li ` (2 subsequent siblings) 6 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong This patch supports attach and detach port in primary and secondary process. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index bc25703490..2e6329c853 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3463,15 +3463,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3502,7 +3499,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3536,10 +3532,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3606,11 +3600,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3774,11 +3766,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3803,8 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3815,6 +3818,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 7d24d25970..080d3a1139 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -306,7 +306,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 9529e16fb6..9216271314 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -765,7 +765,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process 2022-12-06 9:26 ` [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 4:14 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 12:51 UTC (permalink / raw) To: Huisong Li, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen, Aman Singh On 12/6/2022 9:26 AM, Huisong Li wrote: > This patch supports attach and detach port in primary and secondary > process. > Hi Huisong, This patch moves port setup and remove (via alarm callback) to event callback, 1) I can see it is for MP but can you please give more details, what was the problem before, how this solves the issue 2) I am concerned about doing more work on the event callback and observe some unexpected side effects later, can't we handle this out of event callback > Signed-off-by: Huisong Li <lihuisong@huawei.com> > Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> > --- > app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- > app/test-pmd/testpmd.h | 1 - > drivers/net/bonding/bonding_testpmd.c | 1 - > 3 files changed, 22 insertions(+), 18 deletions(-) > > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c > index bc25703490..2e6329c853 100644 > --- a/app/test-pmd/testpmd.c > +++ b/app/test-pmd/testpmd.c > @@ -3463,15 +3463,12 @@ attach_port(char *identifier) > return; > } > > - /* first attach mode: event */ > - if (setup_on_probe_event) { > - /* new ports are detected on RTE_ETH_EVENT_NEW event */ > - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) > - if (ports[pi].port_status == RTE_PORT_HANDLING && > - ports[pi].need_setup != 0) > - setup_attached_port(pi); > + /* > + * first attach mode: event, setting up attached port is done in > + * probing callback. > + */ > + if (setup_on_probe_event) > return; > - } > > /* second attach mode: iterator */ > RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { > @@ -3502,7 +3499,6 @@ setup_attached_port(portid_t pi) > ports_ids[nb_ports++] = pi; > fwd_ports_ids[nb_fwd_ports++] = pi; > nb_cfg_ports = nb_fwd_ports; > - ports[pi].need_setup = 0; > ports[pi].port_status = RTE_PORT_STOPPED; > > printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); > @@ -3536,10 +3532,8 @@ detach_device(struct rte_device *dev) > TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); > return; > } > - remove_invalid_ports(); > > printf("Device is detached\n"); > - printf("Now total ports is %d\n", nb_ports); > printf("Done\n"); > return; > } > @@ -3606,11 +3600,9 @@ detach_devargs(char *identifier) > return; > } > > - remove_invalid_ports(); > - > printf("Device %s is detached\n", identifier); > - printf("Now total ports is %d\n", nb_ports); > printf("Done\n"); > + > rte_devargs_reset(&da); > } > > @@ -3774,11 +3766,22 @@ rmv_port_callback(void *arg) > struct rte_device *device = dev_info.device; > close_port(port_id); > detach_device(device); /* might be already removed or have more ports */ > + remove_invalid_ports(); > + printf("Now total ports is %d\n", nb_ports); > } > if (need_to_start) > start_packet_forwarding(0); > } > > +static void > +remove_invalid_ports_callback(void *arg) > +{ > + RTE_SET_USED(arg); > + > + remove_invalid_ports(); > + printf("Now total ports is %d\n", nb_ports); > +} > + > /* This function is used by the interrupt thread */ > static int > eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, > @@ -3803,8 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, > > switch (type) { > case RTE_ETH_EVENT_NEW: > - ports[port_id].need_setup = 1; > - ports[port_id].port_status = RTE_PORT_HANDLING; > + if (setup_on_probe_event) > + setup_attached_port(port_id); > break; > case RTE_ETH_EVENT_INTR_RMV: > if (rte_eal_alarm_set(100000, > @@ -3815,6 +3818,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, > case RTE_ETH_EVENT_DESTROY: > ports[port_id].port_status = RTE_PORT_CLOSED; > printf("Port %u is closed\n", port_id); > + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, > + (void *)(intptr_t)port_id)) > + fprintf(stderr, "Could not set up deferred device released\n"); > break; > case RTE_ETH_EVENT_RX_AVAIL_THRESH: { > uint16_t rxq_id; > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h > index 7d24d25970..080d3a1139 100644 > --- a/app/test-pmd/testpmd.h > +++ b/app/test-pmd/testpmd.h > @@ -306,7 +306,6 @@ struct rte_port { > uint16_t tx_vlan_id;/**< The tag ID */ > uint16_t tx_vlan_id_outer;/**< The outer tag ID */ > volatile uint16_t port_status; /**< port started or not */ > - uint8_t need_setup; /**< port just attached */ > uint8_t need_reconfig; /**< need reconfiguring port or not */ > uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ > uint8_t rss_flag; /**< enable rss or not */ > diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c > index 9529e16fb6..9216271314 100644 > --- a/drivers/net/bonding/bonding_testpmd.c > +++ b/drivers/net/bonding/bonding_testpmd.c > @@ -765,7 +765,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, > > ports[port_id].update_conf = 1; > ports[port_id].bond_flag = 1; > - ports[port_id].need_setup = 0; > ports[port_id].port_status = RTE_PORT_STOPPED; > } > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process 2023-01-11 12:51 ` Ferruh Yigit @ 2023-01-12 4:14 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-12 4:14 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen, Aman Singh 在 2023/1/11 20:51, Ferruh Yigit 写道: > On 12/6/2022 9:26 AM, Huisong Li wrote: >> This patch supports attach and detach port in primary and secondary >> process. >> > Hi Huisong, > > This patch moves port setup and remove (via alarm callback) to event > callback, > > 1) I can see it is for MP but can you please give more details, what was > the problem before, how this solves the issue The port information in testpmd needs to be updated due to attatching and detaching port, which are done in the same thread as removing or probing device. If testpmd don't support multiple process, it is ok. But it can not work again for multiple process. For example, if primary process does attatch or detach a port, primary and secondary process can receive 'new' or 'destroy' event. And primary can update its port information, but secondary will never update its port information. So we have to move updating port information to event callback. The reason for adding an alarm callback is that the ethdev state is set from 'ATTACHED' to 'UNUSED' only after the event callback finished. The remove_invalid_ports() function removes invalid port according to ethdev state. If we don't add alarm callback, this detached port information can not be removed. > > 2) I am concerned about doing more work on the event callback and > observe some unexpected side effects later, can't we handle this out of > event callback There's no way, if you don't, the port information in testpmd is all messed up. That's very bad. > >> Signed-off-by: Huisong Li <lihuisong@huawei.com> >> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> >> --- >> app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- >> app/test-pmd/testpmd.h | 1 - >> drivers/net/bonding/bonding_testpmd.c | 1 - >> 3 files changed, 22 insertions(+), 18 deletions(-) >> >> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c >> index bc25703490..2e6329c853 100644 >> --- a/app/test-pmd/testpmd.c >> +++ b/app/test-pmd/testpmd.c >> @@ -3463,15 +3463,12 @@ attach_port(char *identifier) >> return; >> } >> >> - /* first attach mode: event */ >> - if (setup_on_probe_event) { >> - /* new ports are detected on RTE_ETH_EVENT_NEW event */ >> - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) >> - if (ports[pi].port_status == RTE_PORT_HANDLING && >> - ports[pi].need_setup != 0) >> - setup_attached_port(pi); >> + /* >> + * first attach mode: event, setting up attached port is done in >> + * probing callback. >> + */ >> + if (setup_on_probe_event) >> return; >> - } >> >> /* second attach mode: iterator */ >> RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { >> @@ -3502,7 +3499,6 @@ setup_attached_port(portid_t pi) >> ports_ids[nb_ports++] = pi; >> fwd_ports_ids[nb_fwd_ports++] = pi; >> nb_cfg_ports = nb_fwd_ports; >> - ports[pi].need_setup = 0; >> ports[pi].port_status = RTE_PORT_STOPPED; >> >> printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >> @@ -3536,10 +3532,8 @@ detach_device(struct rte_device *dev) >> TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); >> return; >> } >> - remove_invalid_ports(); >> >> printf("Device is detached\n"); >> - printf("Now total ports is %d\n", nb_ports); >> printf("Done\n"); >> return; >> } >> @@ -3606,11 +3600,9 @@ detach_devargs(char *identifier) >> return; >> } >> >> - remove_invalid_ports(); >> - >> printf("Device %s is detached\n", identifier); >> - printf("Now total ports is %d\n", nb_ports); >> printf("Done\n"); >> + >> rte_devargs_reset(&da); >> } >> >> @@ -3774,11 +3766,22 @@ rmv_port_callback(void *arg) >> struct rte_device *device = dev_info.device; >> close_port(port_id); >> detach_device(device); /* might be already removed or have more ports */ >> + remove_invalid_ports(); >> + printf("Now total ports is %d\n", nb_ports); >> } >> if (need_to_start) >> start_packet_forwarding(0); >> } >> >> +static void >> +remove_invalid_ports_callback(void *arg) >> +{ >> + RTE_SET_USED(arg); >> + >> + remove_invalid_ports(); >> + printf("Now total ports is %d\n", nb_ports); >> +} >> + >> /* This function is used by the interrupt thread */ >> static int >> eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, >> @@ -3803,8 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, >> >> switch (type) { >> case RTE_ETH_EVENT_NEW: >> - ports[port_id].need_setup = 1; >> - ports[port_id].port_status = RTE_PORT_HANDLING; >> + if (setup_on_probe_event) >> + setup_attached_port(port_id); >> break; >> case RTE_ETH_EVENT_INTR_RMV: >> if (rte_eal_alarm_set(100000, >> @@ -3815,6 +3818,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, >> case RTE_ETH_EVENT_DESTROY: >> ports[port_id].port_status = RTE_PORT_CLOSED; >> printf("Port %u is closed\n", port_id); >> + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, >> + (void *)(intptr_t)port_id)) >> + fprintf(stderr, "Could not set up deferred device released\n"); >> break; >> case RTE_ETH_EVENT_RX_AVAIL_THRESH: { >> uint16_t rxq_id; >> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >> index 7d24d25970..080d3a1139 100644 >> --- a/app/test-pmd/testpmd.h >> +++ b/app/test-pmd/testpmd.h >> @@ -306,7 +306,6 @@ struct rte_port { >> uint16_t tx_vlan_id;/**< The tag ID */ >> uint16_t tx_vlan_id_outer;/**< The outer tag ID */ >> volatile uint16_t port_status; /**< port started or not */ >> - uint8_t need_setup; /**< port just attached */ >> uint8_t need_reconfig; /**< need reconfiguring port or not */ >> uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ >> uint8_t rss_flag; /**< enable rss or not */ >> diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c >> index 9529e16fb6..9216271314 100644 >> --- a/drivers/net/bonding/bonding_testpmd.c >> +++ b/drivers/net/bonding/bonding_testpmd.c >> @@ -765,7 +765,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, >> >> ports[port_id].update_conf = 1; >> ports[port_id].bond_flag = 1; >> - ports[port_id].need_setup = 0; >> ports[port_id].port_status = RTE_PORT_STOPPED; >> } >> > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (3 preceding siblings ...) 2022-12-06 9:26 ` [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2022-12-06 9:26 ` Huisong Li 2023-01-11 12:52 ` Ferruh Yigit 2023-01-09 12:38 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port lihuisong (C) 2023-01-10 16:51 ` Ferruh Yigit 6 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2022-12-06 9:26 UTC (permalink / raw) To: dev, andrew.rybchenko Cc: thomas, ferruh.yigit, liudongdong3, huangdaode, fengchengwen, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 2e6329c853..746f07652a 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3806,6 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3816,6 +3818,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event 2022-12-06 9:26 ` [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2023-01-11 12:52 ` Ferruh Yigit 2023-01-12 4:16 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 12:52 UTC (permalink / raw) To: Huisong Li, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen, Aman Singh On 12/6/2022 9:26 AM, Huisong Li wrote: > When testpmd receives the new or destroy event, the port related > information will be updated. Testpmd must stop packet forwarding > before updating the information to avoid some serious problems. > > Signed-off-by: Huisong Li <lihuisong@huawei.com> > --- > app/test-pmd/testpmd.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c > index 2e6329c853..746f07652a 100644 > --- a/app/test-pmd/testpmd.c > +++ b/app/test-pmd/testpmd.c > @@ -3806,6 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, > > switch (type) { > case RTE_ETH_EVENT_NEW: > + if (test_done == 0) > + stop_packet_forwarding(); testpmd is test application, why not prevent user to issue attach / detach commands when packet forwarding is going on, and force user to stop forwarding explicitly instead of doing this implicitly and silently? Similar to previous comments, as we make things more complex for specific use cases it will be very difficult to update testpmd without hitting unexpected side effects everywhere, at least this is my concern. > if (setup_on_probe_event) > setup_attached_port(port_id); > break; > @@ -3816,6 +3818,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, > "Could not set up deferred device removal\n"); > break; > case RTE_ETH_EVENT_DESTROY: > + if (test_done == 0) > + stop_packet_forwarding(); > ports[port_id].port_status = RTE_PORT_CLOSED; > printf("Port %u is closed\n", port_id); > if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event 2023-01-11 12:52 ` Ferruh Yigit @ 2023-01-12 4:16 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-12 4:16 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen, Aman Singh 在 2023/1/11 20:52, Ferruh Yigit 写道: > On 12/6/2022 9:26 AM, Huisong Li wrote: >> When testpmd receives the new or destroy event, the port related >> information will be updated. Testpmd must stop packet forwarding >> before updating the information to avoid some serious problems. >> >> Signed-off-by: Huisong Li <lihuisong@huawei.com> >> --- >> app/test-pmd/testpmd.c | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c >> index 2e6329c853..746f07652a 100644 >> --- a/app/test-pmd/testpmd.c >> +++ b/app/test-pmd/testpmd.c >> @@ -3806,6 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, >> >> switch (type) { >> case RTE_ETH_EVENT_NEW: >> + if (test_done == 0) >> + stop_packet_forwarding(); > testpmd is test application, why not prevent user to issue attach / > detach commands when packet forwarding is going on, and force user to > stop forwarding explicitly instead of doing this implicitly and silently? For primary process, maybe it is ok by forcing user to stop forwarding explicitly. But for multiple process, it is better to do this before removing or adding. otherwise there'll be a lot of weird prints. > > Similar to previous comments, as we make things more complex for > specific use cases it will be very difficult to update testpmd without > hitting unexpected side effects everywhere, at least this is my concern. Understand your concern. But it's a problem. > >> if (setup_on_probe_event) >> setup_attached_port(port_id); >> break; >> @@ -3816,6 +3818,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, >> "Could not set up deferred device removal\n"); >> break; >> case RTE_ETH_EVENT_DESTROY: >> + if (test_done == 0) >> + stop_packet_forwarding(); >> ports[port_id].port_status = RTE_PORT_CLOSED; >> printf("Port %u is closed\n", port_id); >> if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (4 preceding siblings ...) 2022-12-06 9:26 ` [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2023-01-09 12:38 ` lihuisong (C) 2023-01-10 16:51 ` Ferruh Yigit 6 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-09 12:38 UTC (permalink / raw) To: dev, thomas, ferruh.yigit Cc: liudongdong3, huangdaode, fengchengwen, andrew.rybchenko Hi Thomas and Ferruh, Could you take a look at this patch series? I have modified it according to our last discussion. Best, Huisong 在 2022/12/6 17:26, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v4: fix a misspelling. > -v3: > 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li ` (5 preceding siblings ...) 2023-01-09 12:38 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port lihuisong (C) @ 2023-01-10 16:51 ` Ferruh Yigit 2023-01-11 0:53 ` lihuisong (C) 6 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-10 16:51 UTC (permalink / raw) To: Huisong Li, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 12/6/2022 9:26 AM, Huisong Li wrote: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v4: fix a misspelling. > -v3: > 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > Hi Huisong, I haven't checked the patch in detail yet, but I can see it gives some ABI compatibility warnings, is this expected: 1 function with some indirect sub-type change: [C] 'function int dpaa_eth_eventq_attach(const rte_eth_dev*, int, u16, const rte_event_eth_rx_adapter_queue_conf*)' at dpaa_ethdev.c:1149:1 has some indirect sub-type changes: parameter 1 of type 'const rte_eth_dev*' has sub-type changes: in pointed to type 'const rte_eth_dev': in unqualified underlying type 'struct rte_eth_dev' at ethdev_driver.h:50:1: type size hasn't changed 1 data member change: type of 'rte_eth_dev_state state' changed: type size hasn't changed 1 enumerator insertion: 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' 2 enumerator changes: 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' to '2' at rte_ethdev.h:2000:1 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to '3' at rte_ethdev.h:2000:1 1 function with some indirect sub-type change: [C] 'function int rte_pmd_i40e_set_switch_dev(uint16_t, rte_eth_dev*)' at rte_pmd_i40e.c:3266:1 has some indirect sub-type changes: parameter 2 of type 'rte_eth_dev*' has sub-type changes: in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: type size hasn't changed 1 data member change: type of 'rte_eth_dev_state state' changed: type size hasn't changed 1 enumerator insertion: 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' 2 enumerator changes: 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' to '2' at rte_ethdev.h:2000:1 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to '3' at rte_ethdev.h:2000:1 1 function with some indirect sub-type change: [C] 'function rte_eth_dev* rte_eth_dev_allocate(const char*)' at ethdev_driver.c:72:1 has some indirect sub-type changes: return type changed: in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: type size hasn't changed 1 data member change: type of 'rte_eth_dev_state state' changed: type size hasn't changed 1 enumerator insertion: 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' 2 enumerator changes: 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' to '2' at rte_ethdev.h:2000:1 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to '3' at rte_ethdev.h:2000:1 ... there are more warnings for same issue ... ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-10 16:51 ` Ferruh Yigit @ 2023-01-11 0:53 ` lihuisong (C) 2023-01-11 10:27 ` Ferruh Yigit 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-01-11 0:53 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen 在 2023/1/11 0:51, Ferruh Yigit 写道: > On 12/6/2022 9:26 AM, Huisong Li wrote: >> This patchset fix some bugs and support attaching and detaching port >> in primary and secondary. >> >> --- >> -v4: fix a misspelling. >> -v3: >> 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add modification >> for other bus type. >> 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >> the probelm in patch 2/5. >> -v2: resend due to CI unexplained failure. >> >> Huisong Li (5): >> drivers/bus: restore driver assignment at front of probing >> ethdev: fix skip valid port in probing callback >> app/testpmd: check the validity of the port >> app/testpmd: add attach and detach port for multiple process >> app/testpmd: stop forwarding in new or destroy event >> > Hi Huisong, > > I haven't checked the patch in detail yet, but I can see it gives some > ABI compatibility warnings, is this expected: This is to be expected. Because we insert a device state, RTE_ETH_DEV_ALLOCATED, before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. We may have to announce it. What do you think? > > 1 function with some indirect sub-type change: > > [C] 'function int dpaa_eth_eventq_attach(const rte_eth_dev*, int, u16, > const rte_event_eth_rx_adapter_queue_conf*)' at dpaa_ethdev.c:1149:1 has > some indirect sub-type changes: > parameter 1 of type 'const rte_eth_dev*' has sub-type changes: > in pointed to type 'const rte_eth_dev': > in unqualified underlying type 'struct rte_eth_dev' at > ethdev_driver.h:50:1: > type size hasn't changed > 1 data member change: > type of 'rte_eth_dev_state state' changed: > type size hasn't changed > 1 enumerator insertion: > 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' > 2 enumerator changes: > 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' > to '2' at rte_ethdev.h:2000:1 > 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' > to '3' at rte_ethdev.h:2000:1 > > 1 function with some indirect sub-type change: > > [C] 'function int rte_pmd_i40e_set_switch_dev(uint16_t, rte_eth_dev*)' > at rte_pmd_i40e.c:3266:1 has some indirect sub-type changes: > parameter 2 of type 'rte_eth_dev*' has sub-type changes: > in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: > type size hasn't changed > 1 data member change: > type of 'rte_eth_dev_state state' changed: > type size hasn't changed > 1 enumerator insertion: > 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' > 2 enumerator changes: > 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' > to '2' at rte_ethdev.h:2000:1 > 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to > '3' at rte_ethdev.h:2000:1 > > 1 function with some indirect sub-type change: > > [C] 'function rte_eth_dev* rte_eth_dev_allocate(const char*)' at > ethdev_driver.c:72:1 has some indirect sub-type changes: > return type changed: > in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: > type size hasn't changed > 1 data member change: > type of 'rte_eth_dev_state state' changed: > type size hasn't changed > 1 enumerator insertion: > 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' > 2 enumerator changes: > 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' > to '2' at rte_ethdev.h:2000:1 > 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to > '3' at rte_ethdev.h:2000:1 > > ... there are more warnings for same issue ... > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-11 0:53 ` lihuisong (C) @ 2023-01-11 10:27 ` Ferruh Yigit 2023-01-11 10:46 ` Ferruh Yigit 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 10:27 UTC (permalink / raw) To: lihuisong (C), dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 1/11/2023 12:53 AM, lihuisong (C) wrote: > > 在 2023/1/11 0:51, Ferruh Yigit 写道: >> On 12/6/2022 9:26 AM, Huisong Li wrote: >>> This patchset fix some bugs and support attaching and detaching port >>> in primary and secondary. >>> >>> --- >>> -v4: fix a misspelling. >>> -v3: >>> 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add >>> modification >>> for other bus type. >>> 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >>> the probelm in patch 2/5. >>> -v2: resend due to CI unexplained failure. >>> >>> Huisong Li (5): >>> drivers/bus: restore driver assignment at front of probing >>> ethdev: fix skip valid port in probing callback >>> app/testpmd: check the validity of the port >>> app/testpmd: add attach and detach port for multiple process >>> app/testpmd: stop forwarding in new or destroy event >>> >> Hi Huisong, >> >> I haven't checked the patch in detail yet, but I can see it gives some >> ABI compatibility warnings, is this expected: > This is to be expected. Because we insert a device state, > RTE_ETH_DEV_ALLOCATED, > before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. > We may have to announce it. What do you think? If there is an actual ABI break, it can't go in this release, need to wait LTS release and yes needs deprecation notice in advance. But not all enum value change warnings are real break, need to investigate all warnings one by one. Need to investigate if old application & new dpdk library may cause any unexpected behavior for application. >> >> 1 function with some indirect sub-type change: >> >> [C] 'function int dpaa_eth_eventq_attach(const rte_eth_dev*, int, u16, >> const rte_event_eth_rx_adapter_queue_conf*)' at dpaa_ethdev.c:1149:1 has >> some indirect sub-type changes: >> parameter 1 of type 'const rte_eth_dev*' has sub-type changes: >> in pointed to type 'const rte_eth_dev': >> in unqualified underlying type 'struct rte_eth_dev' at >> ethdev_driver.h:50:1: >> type size hasn't changed >> 1 data member change: >> type of 'rte_eth_dev_state state' changed: >> type size hasn't changed >> 1 enumerator insertion: >> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >> 2 enumerator changes: >> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >> to '2' at rte_ethdev.h:2000:1 >> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' >> to '3' at rte_ethdev.h:2000:1 >> >> 1 function with some indirect sub-type change: >> >> [C] 'function int rte_pmd_i40e_set_switch_dev(uint16_t, rte_eth_dev*)' >> at rte_pmd_i40e.c:3266:1 has some indirect sub-type changes: >> parameter 2 of type 'rte_eth_dev*' has sub-type changes: >> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >> type size hasn't changed >> 1 data member change: >> type of 'rte_eth_dev_state state' changed: >> type size hasn't changed >> 1 enumerator insertion: >> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >> 2 enumerator changes: >> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >> to '2' at rte_ethdev.h:2000:1 >> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >> '3' at rte_ethdev.h:2000:1 >> >> 1 function with some indirect sub-type change: >> >> [C] 'function rte_eth_dev* rte_eth_dev_allocate(const char*)' at >> ethdev_driver.c:72:1 has some indirect sub-type changes: >> return type changed: >> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >> type size hasn't changed >> 1 data member change: >> type of 'rte_eth_dev_state state' changed: >> type size hasn't changed >> 1 enumerator insertion: >> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >> 2 enumerator changes: >> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >> to '2' at rte_ethdev.h:2000:1 >> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >> '3' at rte_ethdev.h:2000:1 >> >> ... there are more warnings for same issue ... >> >> . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-11 10:27 ` Ferruh Yigit @ 2023-01-11 10:46 ` Ferruh Yigit 2023-01-12 2:26 ` lihuisong (C) 2023-01-18 14:12 ` Thomas Monjalon 0 siblings, 2 replies; 95+ messages in thread From: Ferruh Yigit @ 2023-01-11 10:46 UTC (permalink / raw) To: lihuisong (C), dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen On 1/11/2023 10:27 AM, Ferruh Yigit wrote: > On 1/11/2023 12:53 AM, lihuisong (C) wrote: >> >> 在 2023/1/11 0:51, Ferruh Yigit 写道: >>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>> This patchset fix some bugs and support attaching and detaching port >>>> in primary and secondary. >>>> >>>> --- >>>> -v4: fix a misspelling. >>>> -v3: >>>> 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add >>>> modification >>>> for other bus type. >>>> 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >>>> the probelm in patch 2/5. >>>> -v2: resend due to CI unexplained failure. >>>> >>>> Huisong Li (5): >>>> drivers/bus: restore driver assignment at front of probing >>>> ethdev: fix skip valid port in probing callback >>>> app/testpmd: check the validity of the port >>>> app/testpmd: add attach and detach port for multiple process >>>> app/testpmd: stop forwarding in new or destroy event >>>> >>> Hi Huisong, >>> >>> I haven't checked the patch in detail yet, but I can see it gives some >>> ABI compatibility warnings, is this expected: >> This is to be expected. Because we insert a device state, >> RTE_ETH_DEV_ALLOCATED, >> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. >> We may have to announce it. What do you think? > > If there is an actual ABI break, it can't go in this release, need to > wait LTS release and yes needs deprecation notice in advance. > > But not all enum value change warnings are real break, need to > investigate all warnings one by one. > Need to investigate if old application & new dpdk library may cause any > unexpected behavior for application. > OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the enum solves the issue, although logically it won't look nice. Perhaps order can be fixed in next LTS, to have more logical order, but not quite sure if order worth the disturbance may cause in application. >>> >>> 1 function with some indirect sub-type change: >>> >>> [C] 'function int dpaa_eth_eventq_attach(const rte_eth_dev*, int, u16, >>> const rte_event_eth_rx_adapter_queue_conf*)' at dpaa_ethdev.c:1149:1 has >>> some indirect sub-type changes: >>> parameter 1 of type 'const rte_eth_dev*' has sub-type changes: >>> in pointed to type 'const rte_eth_dev': >>> in unqualified underlying type 'struct rte_eth_dev' at >>> ethdev_driver.h:50:1: >>> type size hasn't changed >>> 1 data member change: >>> type of 'rte_eth_dev_state state' changed: >>> type size hasn't changed >>> 1 enumerator insertion: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>> 2 enumerator changes: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>> to '2' at rte_ethdev.h:2000:1 >>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' >>> to '3' at rte_ethdev.h:2000:1 >>> >>> 1 function with some indirect sub-type change: >>> >>> [C] 'function int rte_pmd_i40e_set_switch_dev(uint16_t, rte_eth_dev*)' >>> at rte_pmd_i40e.c:3266:1 has some indirect sub-type changes: >>> parameter 2 of type 'rte_eth_dev*' has sub-type changes: >>> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >>> type size hasn't changed >>> 1 data member change: >>> type of 'rte_eth_dev_state state' changed: >>> type size hasn't changed >>> 1 enumerator insertion: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>> 2 enumerator changes: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>> to '2' at rte_ethdev.h:2000:1 >>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >>> '3' at rte_ethdev.h:2000:1 >>> >>> 1 function with some indirect sub-type change: >>> >>> [C] 'function rte_eth_dev* rte_eth_dev_allocate(const char*)' at >>> ethdev_driver.c:72:1 has some indirect sub-type changes: >>> return type changed: >>> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >>> type size hasn't changed >>> 1 data member change: >>> type of 'rte_eth_dev_state state' changed: >>> type size hasn't changed >>> 1 enumerator insertion: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>> 2 enumerator changes: >>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>> to '2' at rte_ethdev.h:2000:1 >>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >>> '3' at rte_ethdev.h:2000:1 >>> >>> ... there are more warnings for same issue ... >>> >>> . > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-11 10:46 ` Ferruh Yigit @ 2023-01-12 2:26 ` lihuisong (C) 2023-01-18 14:12 ` Thomas Monjalon 1 sibling, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-12 2:26 UTC (permalink / raw) To: Ferruh Yigit, dev, andrew.rybchenko Cc: thomas, liudongdong3, huangdaode, fengchengwen 在 2023/1/11 18:46, Ferruh Yigit 写道: > On 1/11/2023 10:27 AM, Ferruh Yigit wrote: >> On 1/11/2023 12:53 AM, lihuisong (C) wrote: >>> 在 2023/1/11 0:51, Ferruh Yigit 写道: >>>> On 12/6/2022 9:26 AM, Huisong Li wrote: >>>>> This patchset fix some bugs and support attaching and detaching port >>>>> in primary and secondary. >>>>> >>>>> --- >>>>> -v4: fix a misspelling. >>>>> -v3: >>>>> 1) merge patch 1/6 and patch 2/6 into patch 1/5, and add >>>>> modification >>>>> for other bus type. >>>>> 2) add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >>>>> the probelm in patch 2/5. >>>>> -v2: resend due to CI unexplained failure. >>>>> >>>>> Huisong Li (5): >>>>> drivers/bus: restore driver assignment at front of probing >>>>> ethdev: fix skip valid port in probing callback >>>>> app/testpmd: check the validity of the port >>>>> app/testpmd: add attach and detach port for multiple process >>>>> app/testpmd: stop forwarding in new or destroy event >>>>> >>>> Hi Huisong, >>>> >>>> I haven't checked the patch in detail yet, but I can see it gives some >>>> ABI compatibility warnings, is this expected: >>> This is to be expected. Because we insert a device state, >>> RTE_ETH_DEV_ALLOCATED, >>> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. >>> We may have to announce it. What do you think? >> If there is an actual ABI break, it can't go in this release, need to >> wait LTS release and yes needs deprecation notice in advance. >> >> But not all enum value change warnings are real break, need to >> investigate all warnings one by one. >> Need to investigate if old application & new dpdk library may cause any >> unexpected behavior for application. >> > OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the > enum solves the issue, although logically it won't look nice. Thank you for your suggestion, Ferruh. It seems to be ok. I will fix it. > Perhaps order can be fixed in next LTS, to have more logical order, but > not quite sure if order worth the disturbance may cause in application. Probably not worth it. >>>> 1 function with some indirect sub-type change: >>>> >>>> [C] 'function int dpaa_eth_eventq_attach(const rte_eth_dev*, int, u16, >>>> const rte_event_eth_rx_adapter_queue_conf*)' at dpaa_ethdev.c:1149:1 has >>>> some indirect sub-type changes: >>>> parameter 1 of type 'const rte_eth_dev*' has sub-type changes: >>>> in pointed to type 'const rte_eth_dev': >>>> in unqualified underlying type 'struct rte_eth_dev' at >>>> ethdev_driver.h:50:1: >>>> type size hasn't changed >>>> 1 data member change: >>>> type of 'rte_eth_dev_state state' changed: >>>> type size hasn't changed >>>> 1 enumerator insertion: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>>> 2 enumerator changes: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>>> to '2' at rte_ethdev.h:2000:1 >>>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' >>>> to '3' at rte_ethdev.h:2000:1 >>>> >>>> 1 function with some indirect sub-type change: >>>> >>>> [C] 'function int rte_pmd_i40e_set_switch_dev(uint16_t, rte_eth_dev*)' >>>> at rte_pmd_i40e.c:3266:1 has some indirect sub-type changes: >>>> parameter 2 of type 'rte_eth_dev*' has sub-type changes: >>>> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >>>> type size hasn't changed >>>> 1 data member change: >>>> type of 'rte_eth_dev_state state' changed: >>>> type size hasn't changed >>>> 1 enumerator insertion: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>>> 2 enumerator changes: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>>> to '2' at rte_ethdev.h:2000:1 >>>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >>>> '3' at rte_ethdev.h:2000:1 >>>> >>>> 1 function with some indirect sub-type change: >>>> >>>> [C] 'function rte_eth_dev* rte_eth_dev_allocate(const char*)' at >>>> ethdev_driver.c:72:1 has some indirect sub-type changes: >>>> return type changed: >>>> in pointed to type 'struct rte_eth_dev' at ethdev_driver.h:50:1: >>>> type size hasn't changed >>>> 1 data member change: >>>> type of 'rte_eth_dev_state state' changed: >>>> type size hasn't changed >>>> 1 enumerator insertion: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ALLOCATED' value '1' >>>> 2 enumerator changes: >>>> 'rte_eth_dev_state::RTE_ETH_DEV_ATTACHED' from value '1' >>>> to '2' at rte_ethdev.h:2000:1 >>>> 'rte_eth_dev_state::RTE_ETH_DEV_REMOVED' from value '2' to >>>> '3' at rte_ethdev.h:2000:1 >>>> >>>> ... there are more warnings for same issue ... >>>> >>>> . > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-11 10:46 ` Ferruh Yigit 2023-01-12 2:26 ` lihuisong (C) @ 2023-01-18 14:12 ` Thomas Monjalon 2023-01-19 10:31 ` lihuisong (C) 1 sibling, 1 reply; 95+ messages in thread From: Thomas Monjalon @ 2023-01-18 14:12 UTC (permalink / raw) To: lihuisong (C), Ferruh Yigit Cc: dev, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen 11/01/2023 11:46, Ferruh Yigit: > On 1/11/2023 10:27 AM, Ferruh Yigit wrote: > > On 1/11/2023 12:53 AM, lihuisong (C) wrote: > >> 在 2023/1/11 0:51, Ferruh Yigit 写道: > >>> Hi Huisong, > >>> > >>> I haven't checked the patch in detail yet, but I can see it gives some > >>> ABI compatibility warnings, is this expected: > >> This is to be expected. Because we insert a device state, > >> RTE_ETH_DEV_ALLOCATED, > >> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. > >> We may have to announce it. What do you think? > > > > If there is an actual ABI break, it can't go in this release, need to > > wait LTS release and yes needs deprecation notice in advance. > > > > But not all enum value change warnings are real break, need to > > investigate all warnings one by one. > > Need to investigate if old application & new dpdk library may cause any > > unexpected behavior for application. > > > > OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the > enum solves the issue, although logically it won't look nice. > Perhaps order can be fixed in next LTS, to have more logical order, but > not quite sure if order worth the disturbance may cause in application. It is a state with a logical order, so it would be nice to be able to do if (state > RTE_ETH_DEV_ALLOCATED) but given there is RTE_ETH_DEV_REMOVED later in the enum, not sure it is useful. ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-18 14:12 ` Thomas Monjalon @ 2023-01-19 10:31 ` lihuisong (C) 2023-01-19 14:35 ` Thomas Monjalon 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-01-19 10:31 UTC (permalink / raw) To: Thomas Monjalon, Ferruh Yigit Cc: dev, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen 在 2023/1/18 22:12, Thomas Monjalon 写道: > 11/01/2023 11:46, Ferruh Yigit: >> On 1/11/2023 10:27 AM, Ferruh Yigit wrote: >>> On 1/11/2023 12:53 AM, lihuisong (C) wrote: >>>> 在 2023/1/11 0:51, Ferruh Yigit 写道: >>>>> Hi Huisong, >>>>> >>>>> I haven't checked the patch in detail yet, but I can see it gives some >>>>> ABI compatibility warnings, is this expected: >>>> This is to be expected. Because we insert a device state, >>>> RTE_ETH_DEV_ALLOCATED, >>>> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. >>>> We may have to announce it. What do you think? >>> If there is an actual ABI break, it can't go in this release, need to >>> wait LTS release and yes needs deprecation notice in advance. >>> >>> But not all enum value change warnings are real break, need to >>> investigate all warnings one by one. >>> Need to investigate if old application & new dpdk library may cause any >>> unexpected behavior for application. >>> >> OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the >> enum solves the issue, although logically it won't look nice. >> Perhaps order can be fixed in next LTS, to have more logical order, but >> not quite sure if order worth the disturbance may cause in application. > It is a state with a logical order, so it would be nice to be able to do > if (state > RTE_ETH_DEV_ALLOCATED) > but given there is RTE_ETH_DEV_REMOVED later in the enum, not sure it is useful. The device state is internel. Applications should not access it directly, right? Currently, ethdev layer or PMD use it by enum value instead of the way like 'state > RTE_ETH_DEV_ALLOCATED'. But, I encapsulated an API, rte_eth_dev_is_used(), for ethdev or PMD to call. I'm not sure if it can help to eliminate our concerns. > > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-19 10:31 ` lihuisong (C) @ 2023-01-19 14:35 ` Thomas Monjalon 2023-01-28 1:39 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Thomas Monjalon @ 2023-01-19 14:35 UTC (permalink / raw) To: Ferruh Yigit, lihuisong (C) Cc: dev, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen 19/01/2023 11:31, lihuisong (C): > 在 2023/1/18 22:12, Thomas Monjalon 写道: > > 11/01/2023 11:46, Ferruh Yigit: > >> On 1/11/2023 10:27 AM, Ferruh Yigit wrote: > >>> On 1/11/2023 12:53 AM, lihuisong (C) wrote: > >>>> 在 2023/1/11 0:51, Ferruh Yigit 写道: > >>>>> Hi Huisong, > >>>>> > >>>>> I haven't checked the patch in detail yet, but I can see it gives some > >>>>> ABI compatibility warnings, is this expected: > >>>> This is to be expected. Because we insert a device state, > >>>> RTE_ETH_DEV_ALLOCATED, > >>>> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. > >>>> We may have to announce it. What do you think? > >>> If there is an actual ABI break, it can't go in this release, need to > >>> wait LTS release and yes needs deprecation notice in advance. > >>> > >>> But not all enum value change warnings are real break, need to > >>> investigate all warnings one by one. > >>> Need to investigate if old application & new dpdk library may cause any > >>> unexpected behavior for application. > >>> > >> OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the > >> enum solves the issue, although logically it won't look nice. > >> Perhaps order can be fixed in next LTS, to have more logical order, but > >> not quite sure if order worth the disturbance may cause in application. > > It is a state with a logical order, so it would be nice to be able to do > > if (state > RTE_ETH_DEV_ALLOCATED) > > but given there is RTE_ETH_DEV_REMOVED later in the enum, not sure it is useful. > > The device state is internel. Applications should not access it > directly, right? Right > Currently, ethdev layer or PMD use it by enum value instead of the way like > 'state > RTE_ETH_DEV_ALLOCATED'. Right > But, I encapsulated an API, rte_eth_dev_is_used(), for ethdev or PMD to > call. > I'm not sure if it can help to eliminate our concerns. Yes I think it's OK. ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port 2023-01-19 14:35 ` Thomas Monjalon @ 2023-01-28 1:39 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-01-28 1:39 UTC (permalink / raw) To: Thomas Monjalon, Ferruh Yigit Cc: dev, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen 在 2023/1/19 22:35, Thomas Monjalon 写道: > 19/01/2023 11:31, lihuisong (C): >> 在 2023/1/18 22:12, Thomas Monjalon 写道: >>> 11/01/2023 11:46, Ferruh Yigit: >>>> On 1/11/2023 10:27 AM, Ferruh Yigit wrote: >>>>> On 1/11/2023 12:53 AM, lihuisong (C) wrote: >>>>>> 在 2023/1/11 0:51, Ferruh Yigit 写道: >>>>>>> Hi Huisong, >>>>>>> >>>>>>> I haven't checked the patch in detail yet, but I can see it gives some >>>>>>> ABI compatibility warnings, is this expected: >>>>>> This is to be expected. Because we insert a device state, >>>>>> RTE_ETH_DEV_ALLOCATED, >>>>>> before RTE_ETH_DEV_ATTACHED for resolving the issue patch 2/5 mentioned. >>>>>> We may have to announce it. What do you think? >>>>> If there is an actual ABI break, it can't go in this release, need to >>>>> wait LTS release and yes needs deprecation notice in advance. >>>>> >>>>> But not all enum value change warnings are real break, need to >>>>> investigate all warnings one by one. >>>>> Need to investigate if old application & new dpdk library may cause any >>>>> unexpected behavior for application. >>>>> >>>> OR, appending new enum item, `RTE_ETH_DEV_ALLOCATED`, to the end of the >>>> enum solves the issue, although logically it won't look nice. >>>> Perhaps order can be fixed in next LTS, to have more logical order, but >>>> not quite sure if order worth the disturbance may cause in application. >>> It is a state with a logical order, so it would be nice to be able to do >>> if (state > RTE_ETH_DEV_ALLOCATED) >>> but given there is RTE_ETH_DEV_REMOVED later in the enum, not sure it is useful. >> The device state is internel. Applications should not access it >> directly, right? > Right > >> Currently, ethdev layer or PMD use it by enum value instead of the way like >> 'state > RTE_ETH_DEV_ALLOCATED'. > Right > >> But, I encapsulated an API, rte_eth_dev_is_used(), for ethdev or PMD to >> call. >> I'm not sure if it can help to eliminate our concerns. > Yes I think it's OK. ok, I will fix it based on our discussion. > > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> ` (2 preceding siblings ...) 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-01-31 3:33 ` [PATCH V5 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (6 more replies) 2023-05-27 2:11 ` [PATCH V6 " Huisong Li ` (3 subsequent siblings) 7 siblings, 7 replies; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. -v4: fix a misspelling. -v3: #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 1/5] drivers/bus: restore driver assignment at front of probing 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-01-31 3:33 ` [PATCH V5 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (5 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index ff1369353a..13cb3fe0f8 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index bb943b58b5..5f23446f41 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -293,13 +293,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index bc3a7f39fe..efaa1792e9 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 41bc07dde7..2e3f0f2e12 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 8d32d66504..feb1651984 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 2/5] ethdev: fix skip valid port in probing callback 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li 2023-01-31 3:33 ` [PATCH V5 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-05-22 11:04 ` fengchengwen 2023-01-31 3:33 ` [PATCH V5 3/5] app/testpmd: check the validity of the port Huisong Li ` (4 subsequent siblings) 6 siblings, 1 reply; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index b3de490d36..be5c93010e 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6003,7 +6003,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e55be8720e..7f4dafaa22 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3014,7 +3014,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..29e9417bea 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 6a550cfc83..dde3ec84ef 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1542,6 +1542,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 94b8fba5d7..23270ccd73 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index 838b3a8f9f..504bfd99c9 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 5d5e18db1e..86ca303ab5 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -325,7 +325,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -372,7 +372,7 @@ int rte_eth_dev_is_valid_port(uint16_t port_id) { if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) return 0; else return 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index c129ca1eaf..d22de196db 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2000,10 +2000,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 17201fbe0f..094c2a952e 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -327,4 +327,5 @@ INTERNAL { rte_eth_representor_id_get; rte_eth_switch_domain_alloc; rte_eth_switch_domain_free; + rte_eth_dev_is_used; }; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V5 2/5] ethdev: fix skip valid port in probing callback 2023-01-31 3:33 ` [PATCH V5 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2023-05-22 11:04 ` fengchengwen 2023-05-27 1:58 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: fengchengwen @ 2023-05-22 11:04 UTC (permalink / raw) To: Huisong Li, dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode On 2023/1/31 11:33, Huisong Li wrote: > The event callback in application may use the macro RTE_ETH_FOREACH_DEV to > iterate over all enabled ports to do something(like, verifying the port id > validity) when receive a probing event. If the ethdev state of a port is > not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. > > However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing > event. It means that probing callback will skip this port. But this > assignment can not move to front of probing notification. See > commit be8cd210379a ("ethdev: fix port probing notification") ... > > struct rte_eth_dev_sriov { > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map > index 17201fbe0f..094c2a952e 100644 > --- a/lib/ethdev/version.map > +++ b/lib/ethdev/version.map > @@ -327,4 +327,5 @@ INTERNAL { > rte_eth_representor_id_get; > rte_eth_switch_domain_alloc; > rte_eth_switch_domain_free; > + rte_eth_dev_is_used; requires alphabetical order. > }; > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V5 2/5] ethdev: fix skip valid port in probing callback 2023-05-22 11:04 ` fengchengwen @ 2023-05-27 1:58 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-05-27 1:58 UTC (permalink / raw) To: fengchengwen, dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode 在 2023/5/22 19:04, fengchengwen 写道: > On 2023/1/31 11:33, Huisong Li wrote: >> The event callback in application may use the macro RTE_ETH_FOREACH_DEV to >> iterate over all enabled ports to do something(like, verifying the port id >> validity) when receive a probing event. If the ethdev state of a port is >> not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. >> >> However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing >> event. It means that probing callback will skip this port. But this >> assignment can not move to front of probing notification. See >> commit be8cd210379a ("ethdev: fix port probing notification") > ... > >> >> struct rte_eth_dev_sriov { >> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map >> index 17201fbe0f..094c2a952e 100644 >> --- a/lib/ethdev/version.map >> +++ b/lib/ethdev/version.map >> @@ -327,4 +327,5 @@ INTERNAL { >> rte_eth_representor_id_get; >> rte_eth_switch_domain_alloc; >> rte_eth_switch_domain_free; >> + rte_eth_dev_is_used; > requires alphabetical order. Thanks, will fix it in v6. > >> }; >> > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 3/5] app/testpmd: check the validity of the port 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li 2023-01-31 3:33 ` [PATCH V5 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-01-31 3:33 ` [PATCH V5 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-01-31 3:33 ` [PATCH V5 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (3 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index e366f81a0f..ee2383308d 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3798,14 +3798,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 4/5] app/testpmd: add attach and detach port for multiple process 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li ` (2 preceding siblings ...) 2023-01-31 3:33 ` [PATCH V5 3/5] app/testpmd: check the validity of the port Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-01-31 3:33 ` [PATCH V5 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li ` (2 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index ee2383308d..6210982f77 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3463,15 +3463,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3502,7 +3499,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3536,10 +3532,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3606,11 +3600,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3774,11 +3766,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3803,8 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3815,6 +3818,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 7d24d25970..080d3a1139 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -306,7 +306,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index b3c12cada0..160f638e3a 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V5 5/5] app/testpmd: stop forwarding in new or destroy event 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li ` (3 preceding siblings ...) 2023-01-31 3:33 ` [PATCH V5 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2023-01-31 3:33 ` Huisong Li 2023-05-16 11:27 ` [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2023-05-23 0:46 ` fengchengwen 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-01-31 3:33 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 6210982f77..92a185695f 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3806,6 +3806,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3816,6 +3818,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li ` (4 preceding siblings ...) 2023-01-31 3:33 ` [PATCH V5 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2023-05-16 11:27 ` lihuisong (C) 2023-05-23 0:46 ` fengchengwen 6 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-05-16 11:27 UTC (permalink / raw) To: ferruh.yigit, thomas Cc: dev, andrew.rybchenko, liudongdong3, huangdaode, fengchengwen Hi Ferruh and Thomas, Can you continue to take a look at this series? This work has been working on since August last year. /Huisong 在 2023/1/31 11:33, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li ` (5 preceding siblings ...) 2023-05-16 11:27 ` [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) @ 2023-05-23 0:46 ` fengchengwen 6 siblings, 0 replies; 95+ messages in thread From: fengchengwen @ 2023-05-23 0:46 UTC (permalink / raw) To: Huisong Li, dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, huangdaode with 2/5 fixed, Series-acked-by: Chengwen Feng <fengchengwen@huawei.com> On 2023/1/31 11:33, Huisong Li wrote: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> ` (3 preceding siblings ...) 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-05-27 2:11 ` [PATCH V6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (6 more replies) 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li ` (2 subsequent siblings) 7 siblings, 7 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v6: adjust rte_eth_dev_is_used position based on alphabetical order in version.map -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. -v4: fix a misspelling. -v3: #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 1/5] drivers/bus: restore driver assignment at front of probing 2023-05-27 2:11 ` [PATCH V6 " Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-05-27 2:11 ` [PATCH V6 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (5 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index ff1369353a..13cb3fe0f8 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index ffb0c61214..bfe7077645 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -294,13 +294,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index e32a9d517a..ad3ab9e797 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 7974b27295..286ba1fbf9 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 8d32d66504..feb1651984 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 2/5] ethdev: fix skip valid port in probing callback 2023-05-27 2:11 ` [PATCH V6 " Huisong Li 2023-05-27 2:11 ` [PATCH V6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-05-27 2:11 ` [PATCH V6 3/5] app/testpmd: check the validity of the port Huisong Li ` (4 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index ef7b8859d9..74ec0c88fb 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6053,7 +6053,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index a75fa1b7f0..881425bf83 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3145,7 +3145,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..29e9417bea 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 367c0c4878..d5fd6e19ba 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1583,6 +1583,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 94b8fba5d7..23270ccd73 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -164,7 +164,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index b61dae849d..88e56dd9a4 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index d46e74504e..c8f800bb12 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -338,7 +338,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -397,7 +397,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id) int is_valid; if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) is_valid = 0; else is_valid = 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index fe8f7466c8..d4de7942d0 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2002,10 +2002,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 041f0da31f..673123dfb7 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -317,6 +317,7 @@ INTERNAL { rte_eth_dev_get_by_name; rte_eth_dev_is_rx_hairpin_queue; rte_eth_dev_is_tx_hairpin_queue; + rte_eth_dev_is_used; rte_eth_dev_probing_finish; rte_eth_dev_release_port; rte_eth_dev_internal_reset; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 3/5] app/testpmd: check the validity of the port 2023-05-27 2:11 ` [PATCH V6 " Huisong Li 2023-05-27 2:11 ` [PATCH V6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-05-27 2:11 ` [PATCH V6 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-05-27 2:11 ` [PATCH V6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (3 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 5cb6f92523..7e55b1ffc3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3931,14 +3931,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 4/5] app/testpmd: add attach and detach port for multiple process 2023-05-27 2:11 ` [PATCH V6 " Huisong Li ` (2 preceding siblings ...) 2023-05-27 2:11 ` [PATCH V6 3/5] app/testpmd: check the validity of the port Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-05-27 2:11 ` [PATCH V6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li ` (2 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 7e55b1ffc3..f03b3dfb84 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3596,15 +3596,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3635,7 +3632,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3669,10 +3665,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3739,11 +3733,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3907,11 +3899,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3936,8 +3939,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3948,6 +3951,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index bdfbfd36d3..2855dad69e 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -308,7 +308,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index b3c12cada0..160f638e3a 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH V6 5/5] app/testpmd: stop forwarding in new or destroy event 2023-05-27 2:11 ` [PATCH V6 " Huisong Li ` (3 preceding siblings ...) 2023-05-27 2:11 ` [PATCH V6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2023-05-27 2:11 ` Huisong Li 2023-06-06 16:26 ` [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port Ferruh Yigit 2023-07-14 7:21 ` lihuisong (C) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-05-27 2:11 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index f03b3dfb84..7d9d6decc9 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3939,6 +3939,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3949,6 +3951,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port 2023-05-27 2:11 ` [PATCH V6 " Huisong Li ` (4 preceding siblings ...) 2023-05-27 2:11 ` [PATCH V6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2023-06-06 16:26 ` Ferruh Yigit 2023-06-07 10:14 ` lihuisong (C) 2023-07-14 7:21 ` lihuisong (C) 6 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2023-06-06 16:26 UTC (permalink / raw) To: Huisong Li, dev Cc: thomas, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen On 5/27/2023 3:11 AM, Huisong Li wrote: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > Hi Huisong, As commented on v4, I have some concerns on this set. The set does multiple ethdev/testpmd change, but the main target of the patch is not described clearly/simply. It looks like intention is to be able to register NEW event callback in the secondary process and be able to setup device in secondary when primary attaches a device, but my question is why not multi-process communication socket can't be used for this? MP socket/communication/thread is developed for this reason, I am not convinced why it can't be used to sync primary and secondary for device attach/detach. > --- > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port 2023-06-06 16:26 ` [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port Ferruh Yigit @ 2023-06-07 10:14 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-06-07 10:14 UTC (permalink / raw) To: Ferruh Yigit, dev Cc: thomas, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen 在 2023/6/7 0:26, Ferruh Yigit 写道: > On 5/27/2023 3:11 AM, Huisong Li wrote: >> This patchset fix some bugs and support attaching and detaching port >> in primary and secondary. >> > Hi Huisong, > > As commented on v4, I have some concerns on this set. please see my reply. > > The set does multiple ethdev/testpmd change, but the main target of the > patch is not described clearly/simply. The main target is to support attaching and detaching port in primary and secondary. Fixed some problems by the way. > > It looks like intention is to be able to register NEW event callback in > the secondary process and be able to setup device in secondary when > primary attaches a device, > but my question is why not multi-process communication socket can't be > used for this? > > MP socket/communication/thread is developed for this reason, I am not > convinced why it can't be used to sync primary and secondary for device > attach/detach. The secondary process automatically probes the device when primary attaches a device. The primary process automatically probes the device before doing probe phase in secondary when secondary attaches a device. Above behavior itself is attributed to multi-process socket communication(see hogplug_mp.c). This series are just to support this feature in testpmd. But hogplug_mp cannot do something for application, like updating information, which is the duty of application. > > >> --- >> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >> in version.map >> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. >> -v4: fix a misspelling. >> -v3: >> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification >> for other bus type. >> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >> the probelm in patch 2/5. >> -v2: resend due to CI unexplained failure. >> >> Huisong Li (5): >> drivers/bus: restore driver assignment at front of probing >> ethdev: fix skip valid port in probing callback >> app/testpmd: check the validity of the port >> app/testpmd: add attach and detach port for multiple process >> app/testpmd: stop forwarding in new or destroy event >> >> app/test-pmd/testpmd.c | 47 +++++++++++++++--------- >> app/test-pmd/testpmd.h | 1 - >> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >> drivers/bus/pci/pci_common.c | 9 ++++- >> drivers/bus/vdev/vdev.c | 10 ++++- >> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >> drivers/net/bonding/bonding_testpmd.c | 1 - >> drivers/net/mlx5/mlx5.c | 2 +- >> lib/ethdev/ethdev_driver.c | 13 +++++-- >> lib/ethdev/ethdev_driver.h | 12 ++++++ >> lib/ethdev/ethdev_pci.h | 2 +- >> lib/ethdev/rte_class_eth.c | 2 +- >> lib/ethdev/rte_ethdev.c | 4 +- >> lib/ethdev/rte_ethdev.h | 4 +- >> lib/ethdev/version.map | 1 + >> 19 files changed, 114 insertions(+), 44 deletions(-) >> > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port 2023-05-27 2:11 ` [PATCH V6 " Huisong Li ` (5 preceding siblings ...) 2023-06-06 16:26 ` [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port Ferruh Yigit @ 2023-07-14 7:21 ` lihuisong (C) 6 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-07-14 7:21 UTC (permalink / raw) To: dev, ferruh.yigit Cc: thomas, andrew.rybchenko, liudongdong3, liuyonglong, fengchengwen Hi Ferruh, Can you take a look at this series? I added the call stack info for segment fault. /Huisong 在 2023/5/27 10:11, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> ` (4 preceding siblings ...) 2023-05-27 2:11 ` [PATCH V6 " Huisong Li @ 2023-08-02 3:15 ` Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (5 more replies) 2024-01-30 6:36 ` [PATCH v7 " Huisong Li 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li 7 siblings, 6 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:15 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v6: adjust rte_eth_dev_is_used position based on alphabetical order in version.map -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. -v4: fix a misspelling. -v3: #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li @ 2023-08-02 3:15 ` Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (4 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:15 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index 29f99342a7..6313684b8f 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index ffb0c61214..bfe7077645 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -294,13 +294,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 52404ab0fe..c0d9e959ea 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 7974b27295..286ba1fbf9 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 95f3ad78bc..df63eaeaf9 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 2/5] ethdev: fix skip valid port in probing callback 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2023-08-02 3:15 ` Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 3/5] app/testpmd: check the validity of the port Huisong Li ` (3 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:15 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index ee1552452a..bf1910709b 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6108,7 +6108,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index b373306f98..54c6fff889 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3152,7 +3152,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..29e9417bea 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 980f837ab6..5bd2780643 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1582,6 +1582,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 320e3e0093..efe20be1a7 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -165,7 +165,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index b61dae849d..88e56dd9a4 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 0840d2b594..ec44490cde 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -338,7 +338,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -397,7 +397,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id) int is_valid; if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) is_valid = 0; else is_valid = 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 04a2564f22..e7e521efc4 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2003,10 +2003,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index b965d6aa52..ad95329b57 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -326,6 +326,7 @@ INTERNAL { rte_eth_dev_get_by_name; rte_eth_dev_is_rx_hairpin_queue; rte_eth_dev_is_tx_hairpin_queue; + rte_eth_dev_is_used; rte_eth_dev_probing_finish; rte_eth_dev_release_port; rte_eth_dev_internal_reset; -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 3/5] app/testpmd: check the validity of the port 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2023-08-02 3:15 ` Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (2 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:15 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 938ca035d4..f934f6b418 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3945,14 +3945,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 4/5] app/testpmd: add attach and detach port for multiple process 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li ` (2 preceding siblings ...) 2023-08-02 3:15 ` [PATCH RESEND v6 3/5] app/testpmd: check the validity of the port Huisong Li @ 2023-08-02 3:15 ` Huisong Li 2023-08-02 3:16 ` [PATCH RESEND v6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2023-10-09 10:34 ` [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:15 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index f934f6b418..a5124c3231 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3610,15 +3610,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3649,7 +3646,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3683,10 +3679,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3753,11 +3747,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3921,11 +3913,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3950,8 +3953,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3962,6 +3965,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index f1df6a8faf..daacdd7ee4 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -324,7 +324,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index b3c12cada0..160f638e3a 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v6 5/5] app/testpmd: stop forwarding in new or destroy event 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li ` (3 preceding siblings ...) 2023-08-02 3:15 ` [PATCH RESEND v6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2023-08-02 3:16 ` Huisong Li 2023-10-09 10:34 ` [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2023-08-02 3:16 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index a5124c3231..15cbede69b 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3953,6 +3953,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3963,6 +3965,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.22.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li ` (4 preceding siblings ...) 2023-08-02 3:16 ` [PATCH RESEND v6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2023-10-09 10:34 ` lihuisong (C) 2023-10-30 12:17 ` lihuisong (C) 5 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-10-09 10:34 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong Hi Ferruh and Thomas, Can you take a look at this series? They've been over a year on disscussion. 在 2023/8/2 11:15, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port 2023-10-09 10:34 ` [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) @ 2023-10-30 12:17 ` lihuisong (C) 2023-12-08 2:25 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2023-10-30 12:17 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong Hi Ferruh and Thomas, This series have been discussing more than one year. Kindly ping for reivew. 在 2023/10/9 18:34, lihuisong (C) 写道: > Hi Ferruh and Thomas, > > Can you take a look at this series? They've been over a year on > disscussion. > > > 在 2023/8/2 11:15, Huisong Li 写道: >> This patchset fix some bugs and support attaching and detaching port >> in primary and secondary. >> >> --- >> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >> in version.map >> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >> break. >> -v4: fix a misspelling. >> -v3: >> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add >> modification >> for other bus type. >> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >> the probelm in patch 2/5. >> -v2: resend due to CI unexplained failure. >> >> Huisong Li (5): >> drivers/bus: restore driver assignment at front of probing >> ethdev: fix skip valid port in probing callback >> app/testpmd: check the validity of the port >> app/testpmd: add attach and detach port for multiple process >> app/testpmd: stop forwarding in new or destroy event >> >> app/test-pmd/testpmd.c | 47 +++++++++++++++--------- >> app/test-pmd/testpmd.h | 1 - >> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >> drivers/bus/pci/pci_common.c | 9 ++++- >> drivers/bus/vdev/vdev.c | 10 ++++- >> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >> drivers/net/bonding/bonding_testpmd.c | 1 - >> drivers/net/mlx5/mlx5.c | 2 +- >> lib/ethdev/ethdev_driver.c | 13 +++++-- >> lib/ethdev/ethdev_driver.h | 12 ++++++ >> lib/ethdev/ethdev_pci.h | 2 +- >> lib/ethdev/rte_class_eth.c | 2 +- >> lib/ethdev/rte_ethdev.c | 4 +- >> lib/ethdev/rte_ethdev.h | 4 +- >> lib/ethdev/version.map | 1 + >> 19 files changed, 114 insertions(+), 44 deletions(-) >> > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port 2023-10-30 12:17 ` lihuisong (C) @ 2023-12-08 2:25 ` lihuisong (C) 0 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2023-12-08 2:25 UTC (permalink / raw) To: thomas, ferruh.yigit Cc: andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, dev Hi Ferruh and Thomas, This series have been discussing for over a year and a half. Looking back on previous discussions, we have also made some progress and consensus. I am sticking to track it. Because they resolve a real exist issue. Can you take a look at it again? BR, /Huisong 在 2023/10/30 20:17, lihuisong (C) 写道: > Hi Ferruh and Thomas, > > This series have been discussing more than one year. > Kindly ping for reivew. > > > 在 2023/10/9 18:34, lihuisong (C) 写道: >> Hi Ferruh and Thomas, >> >> Can you take a look at this series? They've been over a year on >> disscussion. >> >> >> 在 2023/8/2 11:15, Huisong Li 写道: >>> This patchset fix some bugs and support attaching and detaching port >>> in primary and secondary. >>> >>> --- >>> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >>> in version.map >>> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >>> break. >>> -v4: fix a misspelling. >>> -v3: >>> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add >>> modification >>> for other bus type. >>> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to >>> resolve >>> the probelm in patch 2/5. >>> -v2: resend due to CI unexplained failure. >>> >>> Huisong Li (5): >>> drivers/bus: restore driver assignment at front of probing >>> ethdev: fix skip valid port in probing callback >>> app/testpmd: check the validity of the port >>> app/testpmd: add attach and detach port for multiple process >>> app/testpmd: stop forwarding in new or destroy event >>> >>> app/test-pmd/testpmd.c | 47 >>> +++++++++++++++--------- >>> app/test-pmd/testpmd.h | 1 - >>> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >>> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >>> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >>> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >>> drivers/bus/pci/pci_common.c | 9 ++++- >>> drivers/bus/vdev/vdev.c | 10 ++++- >>> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >>> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >>> drivers/net/bonding/bonding_testpmd.c | 1 - >>> drivers/net/mlx5/mlx5.c | 2 +- >>> lib/ethdev/ethdev_driver.c | 13 +++++-- >>> lib/ethdev/ethdev_driver.h | 12 ++++++ >>> lib/ethdev/ethdev_pci.h | 2 +- >>> lib/ethdev/rte_class_eth.c | 2 +- >>> lib/ethdev/rte_ethdev.c | 4 +- >>> lib/ethdev/rte_ethdev.h | 4 +- >>> lib/ethdev/version.map | 1 + >>> 19 files changed, 114 insertions(+), 44 deletions(-) >>> >> >> . > > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> ` (5 preceding siblings ...) 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-01-30 6:36 ` [PATCH v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (6 more replies) 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li 7 siblings, 7 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v7: fix conflicts -v6: adjust rte_eth_dev_is_used position based on alphabetical order in version.map -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. -v4: fix a misspelling. -v3: #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 1/5] drivers/bus: restore driver assignment at front of probing 2024-01-30 6:36 ` [PATCH v7 " Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-01-30 6:36 ` [PATCH v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (5 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev, Parav Pandit, Xueming Li, Hemant Agrawal, Sachin Saxena, Rosen Xu, Chenbo Xia, Nipun Gupta, Long Li, Thomas Monjalon, Andrew Rybchenko Cc: ferruh.yigit, fengchengwen, liudongdong3, liuyonglong, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index 29f99342a7..6313684b8f 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index ffb0c61214..bfe7077645 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -294,13 +294,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 889a48d2af..404173e7cc 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%04x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 7974b27295..286ba1fbf9 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index b9139c6e6c..0238f15f4f 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -119,6 +119,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -128,9 +134,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 2/5] ethdev: fix skip valid port in probing callback 2024-01-30 6:36 ` [PATCH v7 " Huisong Li 2024-01-30 6:36 ` [PATCH v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-01-30 6:36 ` [PATCH v7 3/5] app/testpmd: check the validity of the port Huisong Li ` (4 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev, Ajit Khaparde, Somnath Kotur, Dariusz Sosnowski, Viacheslav Ovsiienko, Ori Kam, Suanming Mou, Matan Azrad, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Stephen Hemminger Cc: fengchengwen, liudongdong3, liuyonglong, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index acf7e6e46e..cb4c4cfb0e 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6109,7 +6109,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 3a182de248..1f1d9b7d96 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3258,7 +3258,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index bd917a15fc..2b4a6e2b2c 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -52,8 +52,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -217,11 +217,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -239,7 +246,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) if (ret != 0) return ret; - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index f05f68a67c..238bd72ab0 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1596,6 +1596,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 737fff1833..c07a98e04f 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -165,7 +165,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index bc003db8af..31cabee525 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index f61dc7dd7b..b6b7a20e35 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -349,7 +349,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -408,7 +408,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id) int is_valid; if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) is_valid = 0; else is_valid = 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 2687c23fa6..5079fea8a6 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2068,10 +2068,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index a050baab0f..26867eb6e5 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -333,6 +333,7 @@ INTERNAL { rte_eth_dev_get_by_name; rte_eth_dev_is_rx_hairpin_queue; rte_eth_dev_is_tx_hairpin_queue; + rte_eth_dev_is_used; rte_eth_dev_probing_finish; rte_eth_dev_release_port; rte_eth_dev_internal_reset; -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 3/5] app/testpmd: check the validity of the port 2024-01-30 6:36 ` [PATCH v7 " Huisong Li 2024-01-30 6:36 ` [PATCH v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2024-01-30 6:36 ` [PATCH v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-01-30 6:36 ` [PATCH v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (3 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev, Aman Singh, Yuying Zhang Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 9e4e99e53b..859d4aad7d 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3960,14 +3960,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 4/5] app/testpmd: add attach and detach port for multiple process 2024-01-30 6:36 ` [PATCH v7 " Huisong Li ` (2 preceding siblings ...) 2024-01-30 6:36 ` [PATCH v7 3/5] app/testpmd: check the validity of the port Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-01-30 6:36 ` [PATCH v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li ` (2 subsequent siblings) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev, Aman Singh, Yuying Zhang, Chas Williams, Min Hu (Connor) Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 859d4aad7d..db985d33a3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3625,15 +3625,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3664,7 +3661,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3698,10 +3694,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3768,11 +3762,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3936,11 +3928,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3965,8 +3968,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3977,6 +3980,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 9b10a9ea1c..0e860c2e67 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -326,7 +326,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..f4845206ad 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonding_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH v7 5/5] app/testpmd: stop forwarding in new or destroy event 2024-01-30 6:36 ` [PATCH v7 " Huisong Li ` (3 preceding siblings ...) 2024-01-30 6:36 ` [PATCH v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2024-01-30 6:36 ` Huisong Li 2024-03-08 10:38 ` [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2024-04-23 11:17 ` lihuisong (C) 6 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-01-30 6:36 UTC (permalink / raw) To: dev, Aman Singh, Yuying Zhang Cc: thomas, ferruh.yigit, andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index db985d33a3..dfdd7e723d 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3968,6 +3968,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3978,6 +3980,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port 2024-01-30 6:36 ` [PATCH v7 " Huisong Li ` (4 preceding siblings ...) 2024-01-30 6:36 ` [PATCH v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2024-03-08 10:38 ` lihuisong (C) 2024-04-23 11:17 ` lihuisong (C) 6 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2024-03-08 10:38 UTC (permalink / raw) To: thomas, ferruh.yigit Cc: andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong, dev Hi Ferruh and Thomas, Kindly ping for review. 在 2024/1/30 14:36, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v7: fix conflicts > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port 2024-01-30 6:36 ` [PATCH v7 " Huisong Li ` (5 preceding siblings ...) 2024-03-08 10:38 ` [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) @ 2024-04-23 11:17 ` lihuisong (C) 6 siblings, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2024-04-23 11:17 UTC (permalink / raw) To: dev, thomas, ferruh.yigit Cc: andrew.rybchenko, fengchengwen, liudongdong3, liuyonglong Hi Ferruh and Thomas, It's been almost two years since this issue was reported. We have discussed a lot before, and also made some progress and consensus. Can you take a look at it again? Looking forward to your reply. BR/ Huisong 在 2024/1/30 14:36, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v7: fix conflicts > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port [not found] <20220825024425.10534-1-lihuisong@huawei.com> ` (6 preceding siblings ...) 2024-01-30 6:36 ` [PATCH v7 " Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li ` (5 more replies) 7 siblings, 6 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko Cc: dev, fengchengwen, liuyonglong, lihuisong This patchset fix some bugs and support attaching and detaching port in primary and secondary. --- -v7: fix conflicts -v6: adjust rte_eth_dev_is_used position based on alphabetical order in version.map -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. -v4: fix a misspelling. -v3: #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification for other bus type. #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve the probelm in patch 2/5. -v2: resend due to CI unexplained failure. Huisong Li (5): drivers/bus: restore driver assignment at front of probing ethdev: fix skip valid port in probing callback app/testpmd: check the validity of the port app/testpmd: add attach and detach port for multiple process app/testpmd: stop forwarding in new or destroy event app/test-pmd/testpmd.c | 47 +++++++++++++++--------- app/test-pmd/testpmd.h | 1 - drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- drivers/bus/dpaa/dpaa_bus.c | 9 ++++- drivers/bus/fslmc/fslmc_bus.c | 8 +++- drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- drivers/bus/pci/pci_common.c | 9 ++++- drivers/bus/vdev/vdev.c | 10 ++++- drivers/bus/vmbus/vmbus_common.c | 9 ++++- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bonding/bonding_testpmd.c | 1 - drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 +++++-- lib/ethdev/ethdev_driver.h | 12 ++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 +- lib/ethdev/rte_ethdev.h | 4 +- lib/ethdev/version.map | 1 + 19 files changed, 114 insertions(+), 44 deletions(-) -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 1/5] drivers/bus: restore driver assignment at front of probing 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li ` (4 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko, Parav Pandit, Xueming Li, Hemant Agrawal, Sachin Saxena, Rosen Xu, Chenbo Xia, Nipun Gupta, Long Li Cc: dev, fengchengwen, liuyonglong, lihuisong The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index e6cbc4d356..8ce41a38d9 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index 1f6997c77e..eae2a03ff3 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -697,17 +697,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index c155f4a2fd..db13b1d60c 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -472,15 +472,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index ffb0c61214..bfe7077645 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -294,13 +294,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 1173f0887c..0a237a04cf 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -297,6 +297,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } PCI_LOG(INFO, "Probe PCI driver: %s (%x:%04x) device: "PCI_PRI_FMT" (socket %i)", @@ -309,6 +315,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -320,8 +327,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index ec7abe7cda..2bf8e1f7fe 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index b9139c6e6c..0238f15f4f 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -119,6 +119,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -128,9 +134,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 3/5] app/testpmd: check the validity of the port Huisong Li ` (3 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko, Ajit Khaparde, Somnath Kotur, Dariusz Sosnowski, Viacheslav Ovsiienko, Ori Kam, Suanming Mou, Matan Azrad, Stephen Hemminger Cc: dev, fengchengwen, liuyonglong, lihuisong The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index c6ad764813..7401dcd8b5 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6612,7 +6612,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 8d266b0e64..0df49e1f69 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3371,7 +3371,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index c335a25a82..a87dbb00ff 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -55,8 +55,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -221,11 +221,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -243,7 +250,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) if (ret != 0) return ret; - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index abed4784aa..aa35b65848 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1704,6 +1704,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index ec4f731270..05dec6716b 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -179,7 +179,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index b52f1dd9f2..81e70670d9 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index a1f7efa913..4dc66abb7b 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -349,7 +349,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -408,7 +408,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id) int is_valid; if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) is_valid = 0; else is_valid = 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index a9f92006da..9cc37e8cde 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2083,10 +2083,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index f63dc32aa2..6ecf1ab89d 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -349,6 +349,7 @@ INTERNAL { rte_eth_dev_get_by_name; rte_eth_dev_is_rx_hairpin_queue; rte_eth_dev_is_tx_hairpin_queue; + rte_eth_dev_is_used; rte_eth_dev_probing_finish; rte_eth_dev_release_port; rte_eth_dev_internal_reset; -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 3/5] app/testpmd: check the validity of the port 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li ` (2 subsequent siblings) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko, Aman Singh, Yuying Zhang Cc: dev, fengchengwen, liuyonglong, lihuisong This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Aman Singh <aman.deep.singh@intel.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index b1401136e4..b577084795 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3957,14 +3957,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 4/5] app/testpmd: add attach and detach port for multiple process 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li ` (2 preceding siblings ...) 2024-09-29 5:52 ` [PATCH RESEND v7 3/5] app/testpmd: check the validity of the port Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2024-10-08 2:32 ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko, Aman Singh, Yuying Zhang, Chas Williams, Min Hu (Connor) Cc: dev, fengchengwen, liuyonglong, lihuisong The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index b577084795..7aaf71f157 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3625,15 +3625,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3664,7 +3661,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3698,10 +3694,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3768,11 +3762,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3933,11 +3925,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3962,8 +3965,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3974,6 +3977,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index f9ab88d667..7dd87b9b6f 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -327,7 +327,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..f4845206ad 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonding_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* [PATCH RESEND v7 5/5] app/testpmd: stop forwarding in new or destroy event 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li ` (3 preceding siblings ...) 2024-09-29 5:52 ` [PATCH RESEND v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li @ 2024-09-29 5:52 ` Huisong Li 2024-10-08 2:32 ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 5 siblings, 0 replies; 95+ messages in thread From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko, Aman Singh, Yuying Zhang Cc: dev, fengchengwen, liuyonglong, lihuisong When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li <lihuisong@huawei.com> Acked-by: Chengwen Feng <fengchengwen@huawei.com> --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 7aaf71f157..60e85ed067 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3965,6 +3965,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3975,6 +3977,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, -- 2.33.0 ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li ` (4 preceding siblings ...) 2024-09-29 5:52 ` [PATCH RESEND v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li @ 2024-10-08 2:32 ` lihuisong (C) 2024-10-18 1:04 ` Ferruh Yigit 5 siblings, 1 reply; 95+ messages in thread From: lihuisong (C) @ 2024-10-08 2:32 UTC (permalink / raw) To: thomas, ferruh.yigit, andrew.rybchenko; +Cc: dev, fengchengwen, liuyonglong Hi Thomas and Ferruh, We've discussed it on and off a few times, and we've reached some consensus. They've been going through more than 2 years😅 Can you have a look at this series again? If we really don't need it, I will drop it from my upstreaming list. /Huisong 在 2024/9/29 13:52, Huisong Li 写道: > This patchset fix some bugs and support attaching and detaching port > in primary and secondary. > > --- > -v7: fix conflicts > -v6: adjust rte_eth_dev_is_used position based on alphabetical order > in version.map > -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break. > -v4: fix a misspelling. > -v3: > #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification > for other bus type. > #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve > the probelm in patch 2/5. > -v2: resend due to CI unexplained failure. > > Huisong Li (5): > drivers/bus: restore driver assignment at front of probing > ethdev: fix skip valid port in probing callback > app/testpmd: check the validity of the port > app/testpmd: add attach and detach port for multiple process > app/testpmd: stop forwarding in new or destroy event > > app/test-pmd/testpmd.c | 47 +++++++++++++++--------- > app/test-pmd/testpmd.h | 1 - > drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- > drivers/bus/dpaa/dpaa_bus.c | 9 ++++- > drivers/bus/fslmc/fslmc_bus.c | 8 +++- > drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- > drivers/bus/pci/pci_common.c | 9 ++++- > drivers/bus/vdev/vdev.c | 10 ++++- > drivers/bus/vmbus/vmbus_common.c | 9 ++++- > drivers/net/bnxt/bnxt_ethdev.c | 3 +- > drivers/net/bonding/bonding_testpmd.c | 1 - > drivers/net/mlx5/mlx5.c | 2 +- > lib/ethdev/ethdev_driver.c | 13 +++++-- > lib/ethdev/ethdev_driver.h | 12 ++++++ > lib/ethdev/ethdev_pci.h | 2 +- > lib/ethdev/rte_class_eth.c | 2 +- > lib/ethdev/rte_ethdev.c | 4 +- > lib/ethdev/rte_ethdev.h | 4 +- > lib/ethdev/version.map | 1 + > 19 files changed, 114 insertions(+), 44 deletions(-) > ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port 2024-10-08 2:32 ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) @ 2024-10-18 1:04 ` Ferruh Yigit 2024-10-18 2:48 ` lihuisong (C) 0 siblings, 1 reply; 95+ messages in thread From: Ferruh Yigit @ 2024-10-18 1:04 UTC (permalink / raw) To: lihuisong (C), thomas, andrew.rybchenko; +Cc: dev, fengchengwen, liuyonglong On 10/8/2024 3:32 AM, lihuisong (C) wrote: > Hi Thomas and Ferruh, > > We've discussed it on and off a few times, and we've reached some > consensus. > They've been going through more than 2 years😅 > Can you have a look at this series again? > If we really don't need it, I will drop it from my upstreaming list. > Hi Huisong, I was not really convinced with the patch series, but did not want to block it outright, sorry that this caused patch series stay around. As checked again, still feels like adding unnecessary complexity, and I am for rejecting this series. Overall target is to be able to support hotplug with primary/secondary process, and uses event handlers for this but this requires adding a new ethdev state to be able iterate over devices etc... Perhaps better way to support this without relying on event handlers. > /Huisong > > > 在 2024/9/29 13:52, Huisong Li 写道: >> This patchset fix some bugs and support attaching and detaching port >> in primary and secondary. >> >> --- >> -v7: fix conflicts >> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >> in version.map >> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >> break. >> -v4: fix a misspelling. >> -v3: >> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification >> for other bus type. >> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >> the probelm in patch 2/5. >> -v2: resend due to CI unexplained failure. >> >> Huisong Li (5): >> drivers/bus: restore driver assignment at front of probing >> ethdev: fix skip valid port in probing callback >> app/testpmd: check the validity of the port >> app/testpmd: add attach and detach port for multiple process >> app/testpmd: stop forwarding in new or destroy event >> >> app/test-pmd/testpmd.c | 47 +++++++++++++++--------- >> app/test-pmd/testpmd.h | 1 - >> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >> drivers/bus/pci/pci_common.c | 9 ++++- >> drivers/bus/vdev/vdev.c | 10 ++++- >> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >> drivers/net/bonding/bonding_testpmd.c | 1 - >> drivers/net/mlx5/mlx5.c | 2 +- >> lib/ethdev/ethdev_driver.c | 13 +++++-- >> lib/ethdev/ethdev_driver.h | 12 ++++++ >> lib/ethdev/ethdev_pci.h | 2 +- >> lib/ethdev/rte_class_eth.c | 2 +- >> lib/ethdev/rte_ethdev.c | 4 +- >> lib/ethdev/rte_ethdev.h | 4 +- >> lib/ethdev/version.map | 1 + >> 19 files changed, 114 insertions(+), 44 deletions(-) >> ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port 2024-10-18 1:04 ` Ferruh Yigit @ 2024-10-18 2:48 ` lihuisong (C) 2024-10-26 4:11 ` lihuisong (C) 2024-10-29 22:12 ` Ferruh Yigit 0 siblings, 2 replies; 95+ messages in thread From: lihuisong (C) @ 2024-10-18 2:48 UTC (permalink / raw) To: Ferruh Yigit, thomas, andrew.rybchenko; +Cc: dev, fengchengwen, liuyonglong Hi Ferruh, Thanks for your considering again. please see reply inline. 在 2024/10/18 9:04, Ferruh Yigit 写道: > On 10/8/2024 3:32 AM, lihuisong (C) wrote: >> Hi Thomas and Ferruh, >> >> We've discussed it on and off a few times, and we've reached some >> consensus. >> They've been going through more than 2 years😅 >> Can you have a look at this series again? >> If we really don't need it, I will drop it from my upstreaming list. >> > Hi Huisong, > > I was not really convinced with the patch series, but did not want to > block it outright, sorry that this caused patch series stay around. > > As checked again, still feels like adding unnecessary complexity, and I > am for rejecting this series. > > Overall target is to be able to support hotplug with primary/secondary > process, and uses event handlers for this but this requires adding a new > ethdev state to be able iterate over devices etc... > Perhaps better way to support this without relying on event handlers. Ignoring the modification of tesptmd is ok to me. But we need to restrict testpmd not to support attach and detach port in multiple process case. Otherwise. these issues this series solved will be encountered. BTW, I want to say the patch [2/5] which introduced RTE_ETH_DEV_ALLOCATED should be thought again. Because it is an real issue in ethdev layer. This is also the fruit that Thomas, you and I discussed before. Please look at this patch again. /Huisong > > >> /Huisong >> >> >> 在 2024/9/29 13:52, Huisong Li 写道: >>> This patchset fix some bugs and support attaching and detaching port >>> in primary and secondary. >>> >>> --- >>> -v7: fix conflicts >>> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >>> in version.map >>> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >>> break. >>> -v4: fix a misspelling. >>> -v3: >>> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification >>> for other bus type. >>> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve >>> the probelm in patch 2/5. >>> -v2: resend due to CI unexplained failure. >>> >>> Huisong Li (5): >>> drivers/bus: restore driver assignment at front of probing >>> ethdev: fix skip valid port in probing callback >>> app/testpmd: check the validity of the port >>> app/testpmd: add attach and detach port for multiple process >>> app/testpmd: stop forwarding in new or destroy event >>> >>> app/test-pmd/testpmd.c | 47 +++++++++++++++--------- >>> app/test-pmd/testpmd.h | 1 - >>> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >>> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >>> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >>> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >>> drivers/bus/pci/pci_common.c | 9 ++++- >>> drivers/bus/vdev/vdev.c | 10 ++++- >>> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >>> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >>> drivers/net/bonding/bonding_testpmd.c | 1 - >>> drivers/net/mlx5/mlx5.c | 2 +- >>> lib/ethdev/ethdev_driver.c | 13 +++++-- >>> lib/ethdev/ethdev_driver.h | 12 ++++++ >>> lib/ethdev/ethdev_pci.h | 2 +- >>> lib/ethdev/rte_class_eth.c | 2 +- >>> lib/ethdev/rte_ethdev.c | 4 +- >>> lib/ethdev/rte_ethdev.h | 4 +- >>> lib/ethdev/version.map | 1 + >>> 19 files changed, 114 insertions(+), 44 deletions(-) >>> > . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port 2024-10-18 2:48 ` lihuisong (C) @ 2024-10-26 4:11 ` lihuisong (C) 2024-10-29 22:12 ` Ferruh Yigit 1 sibling, 0 replies; 95+ messages in thread From: lihuisong (C) @ 2024-10-26 4:11 UTC (permalink / raw) To: Ferruh Yigit; +Cc: dev, fengchengwen, liuyonglong, thomas, andrew.rybchenko Hi Ferruh, 在 2024/10/18 10:48, lihuisong (C) 写道: > Hi Ferruh, > > Thanks for your considering again. please see reply inline. > > 在 2024/10/18 9:04, Ferruh Yigit 写道: >> On 10/8/2024 3:32 AM, lihuisong (C) wrote: >>> Hi Thomas and Ferruh, >>> >>> We've discussed it on and off a few times, and we've reached some >>> consensus. >>> They've been going through more than 2 years😅 >>> Can you have a look at this series again? >>> If we really don't need it, I will drop it from my upstreaming list. >>> >> Hi Huisong, >> >> I was not really convinced with the patch series, but did not want to >> block it outright, sorry that this caused patch series stay around. >> >> As checked again, still feels like adding unnecessary complexity, and I >> am for rejecting this series. >> >> Overall target is to be able to support hotplug with primary/secondary >> process, and uses event handlers for this but this requires adding a new >> ethdev state to be able iterate over devices etc... >> Perhaps better way to support this without relying on event handlers. > Ignoring the modification of tesptmd is ok to me. > But we need to restrict testpmd not to support attach and detach port > in multiple process case. > Otherwise. these issues this series solved will be encountered. > > BTW, I want to say the patch [2/5] which introduced > RTE_ETH_DEV_ALLOCATED should be thought again. > Because it is an real issue in ethdev layer. This is also the fruit > that Thomas, you and I discussed before. > Please look at this patch again. Can you please take a look at my above reply? > > /Huisong >> >> >>> /Huisong >>> >>> >>> 在 2024/9/29 13:52, Huisong Li 写道: >>>> This patchset fix some bugs and support attaching and detaching port >>>> in primary and secondary. >>>> >>>> --- >>>> -v7: fix conflicts >>>> -v6: adjust rte_eth_dev_is_used position based on alphabetical >>>> order >>>> in version.map >>>> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >>>> break. >>>> -v4: fix a misspelling. >>>> -v3: >>>> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add >>>> modification >>>> for other bus type. >>>> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to >>>> resolve >>>> the probelm in patch 2/5. >>>> -v2: resend due to CI unexplained failure. >>>> >>>> Huisong Li (5): >>>> drivers/bus: restore driver assignment at front of probing >>>> ethdev: fix skip valid port in probing callback >>>> app/testpmd: check the validity of the port >>>> app/testpmd: add attach and detach port for multiple process >>>> app/testpmd: stop forwarding in new or destroy event >>>> >>>> app/test-pmd/testpmd.c | 47 >>>> +++++++++++++++--------- >>>> app/test-pmd/testpmd.h | 1 - >>>> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >>>> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >>>> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >>>> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >>>> drivers/bus/pci/pci_common.c | 9 ++++- >>>> drivers/bus/vdev/vdev.c | 10 ++++- >>>> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >>>> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >>>> drivers/net/bonding/bonding_testpmd.c | 1 - >>>> drivers/net/mlx5/mlx5.c | 2 +- >>>> lib/ethdev/ethdev_driver.c | 13 +++++-- >>>> lib/ethdev/ethdev_driver.h | 12 ++++++ >>>> lib/ethdev/ethdev_pci.h | 2 +- >>>> lib/ethdev/rte_class_eth.c | 2 +- >>>> lib/ethdev/rte_ethdev.c | 4 +- >>>> lib/ethdev/rte_ethdev.h | 4 +- >>>> lib/ethdev/version.map | 1 + >>>> 19 files changed, 114 insertions(+), 44 deletions(-) >>>> >> . ^ permalink raw reply [flat|nested] 95+ messages in thread
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port 2024-10-18 2:48 ` lihuisong (C) 2024-10-26 4:11 ` lihuisong (C) @ 2024-10-29 22:12 ` Ferruh Yigit 1 sibling, 0 replies; 95+ messages in thread From: Ferruh Yigit @ 2024-10-29 22:12 UTC (permalink / raw) To: lihuisong (C), thomas, andrew.rybchenko, Stephen Hemminger Cc: dev, fengchengwen, liuyonglong On 10/18/2024 3:48 AM, lihuisong (C) wrote: > Hi Ferruh, > > Thanks for your considering again. please see reply inline. > > 在 2024/10/18 9:04, Ferruh Yigit 写道: >> On 10/8/2024 3:32 AM, lihuisong (C) wrote: >>> Hi Thomas and Ferruh, >>> >>> We've discussed it on and off a few times, and we've reached some >>> consensus. >>> They've been going through more than 2 years😅 >>> Can you have a look at this series again? >>> If we really don't need it, I will drop it from my upstreaming list. >>> >> Hi Huisong, >> >> I was not really convinced with the patch series, but did not want to >> block it outright, sorry that this caused patch series stay around. >> >> As checked again, still feels like adding unnecessary complexity, and I >> am for rejecting this series. >> >> Overall target is to be able to support hotplug with primary/secondary >> process, and uses event handlers for this but this requires adding a new >> ethdev state to be able iterate over devices etc... >> Perhaps better way to support this without relying on event handlers. > Ignoring the modification of tesptmd is ok to me. > But we need to restrict testpmd not to support attach and detach port in > multiple process case. > Otherwise. these issues this series solved will be encountered. > > BTW, I want to say the patch [2/5] which introduced > RTE_ETH_DEV_ALLOCATED should be thought again. > Because it is an real issue in ethdev layer. This is also the fruit that > Thomas, you and I discussed before. > Please look at this patch again. > RTE_ETH_DEV_ALLOCATED is added to run RTE_ETH_FOREACH_DEV in the event handler, more specifically on the 'RTE_ETH_EVENT_NEW' event handler, right? Without testpmd event handler update, what is the reason/usecase for above ethdev change? Thomas, Andrew, Stephen, please feel free to chime in. > /Huisong >> >> >>> /Huisong >>> >>> >>> 在 2024/9/29 13:52, Huisong Li 写道: >>>> This patchset fix some bugs and support attaching and detaching port >>>> in primary and secondary. >>>> >>>> --- >>>> -v7: fix conflicts >>>> -v6: adjust rte_eth_dev_is_used position based on alphabetical order >>>> in version.map >>>> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi >>>> break. >>>> -v4: fix a misspelling. >>>> -v3: >>>> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add >>>> modification >>>> for other bus type. >>>> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to >>>> resolve >>>> the probelm in patch 2/5. >>>> -v2: resend due to CI unexplained failure. >>>> >>>> Huisong Li (5): >>>> drivers/bus: restore driver assignment at front of probing >>>> ethdev: fix skip valid port in probing callback >>>> app/testpmd: check the validity of the port >>>> app/testpmd: add attach and detach port for multiple process >>>> app/testpmd: stop forwarding in new or destroy event >>>> >>>> app/test-pmd/testpmd.c | 47 ++++++++++++++ >>>> +--------- >>>> app/test-pmd/testpmd.h | 1 - >>>> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++- >>>> drivers/bus/dpaa/dpaa_bus.c | 9 ++++- >>>> drivers/bus/fslmc/fslmc_bus.c | 8 +++- >>>> drivers/bus/ifpga/ifpga_bus.c | 12 ++++-- >>>> drivers/bus/pci/pci_common.c | 9 ++++- >>>> drivers/bus/vdev/vdev.c | 10 ++++- >>>> drivers/bus/vmbus/vmbus_common.c | 9 ++++- >>>> drivers/net/bnxt/bnxt_ethdev.c | 3 +- >>>> drivers/net/bonding/bonding_testpmd.c | 1 - >>>> drivers/net/mlx5/mlx5.c | 2 +- >>>> lib/ethdev/ethdev_driver.c | 13 +++++-- >>>> lib/ethdev/ethdev_driver.h | 12 ++++++ >>>> lib/ethdev/ethdev_pci.h | 2 +- >>>> lib/ethdev/rte_class_eth.c | 2 +- >>>> lib/ethdev/rte_ethdev.c | 4 +- >>>> lib/ethdev/rte_ethdev.h | 4 +- >>>> lib/ethdev/version.map | 1 + >>>> 19 files changed, 114 insertions(+), 44 deletions(-) >>>> >> . ^ permalink raw reply [flat|nested] 95+ messages in thread
end of thread, other threads:[~2024-10-29 22:12 UTC | newest] Thread overview: 95+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20220825024425.10534-1-lihuisong@huawei.com> 2022-09-15 12:45 ` [PATCH V2 0/6] [PATCH 0/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-09-15 12:45 ` [PATCH V2 1/6] bus/pci: fix a segfault when call callback Huisong Li 2022-10-10 19:49 ` Thomas Monjalon 2022-10-25 3:25 ` lihuisong (C) 2022-09-15 12:45 ` [PATCH V2 2/6] bus/vdev: " Huisong Li 2022-09-15 12:45 ` [PATCH V2 3/6] ethdev: fix push new event Huisong Li 2022-09-27 10:49 ` Thomas Monjalon 2022-10-08 4:09 ` lihuisong (C) 2022-10-25 3:26 ` lihuisong (C) 2022-09-15 12:45 ` [PATCH V2 4/6] app/testpmd: check the validity of the port Huisong Li 2022-09-22 5:07 ` Singh, Aman Deep 2022-09-15 12:45 ` [PATCH V2 5/6] app/testpmd: support attach and detach port for MP Huisong Li 2022-09-15 12:45 ` [PATCH V2 6/6] app/testpmd: stop packet forwarding in new and destroy event Huisong Li 2022-12-06 6:45 ` [PATCH V3 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 6:45 ` [PATCH V3 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2022-12-06 6:45 ` [PATCH V3 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2022-12-06 6:45 ` [PATCH V3 3/5] app/testpmd: check the validity of the port Huisong Li 2022-12-06 6:45 ` [PATCH V3 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2022-12-06 6:45 ` [PATCH V3 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2022-12-06 9:26 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port Huisong Li 2022-12-06 9:26 ` [PATCH V4 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 2:44 ` lihuisong (C) 2023-02-15 16:09 ` Ferruh Yigit 2023-02-28 2:21 ` lihuisong (C) 2023-06-06 16:12 ` Ferruh Yigit 2023-06-07 10:11 ` lihuisong (C) 2023-06-15 2:21 ` lihuisong (C) 2022-12-06 9:26 ` [PATCH V4 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 4:12 ` lihuisong (C) 2022-12-06 9:26 ` [PATCH V4 3/5] app/testpmd: check the validity of the port Huisong Li 2022-12-06 9:26 ` [PATCH V4 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2023-01-11 12:51 ` Ferruh Yigit 2023-01-12 4:14 ` lihuisong (C) 2022-12-06 9:26 ` [PATCH V4 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2023-01-11 12:52 ` Ferruh Yigit 2023-01-12 4:16 ` lihuisong (C) 2023-01-09 12:38 ` [PATCH V4 0/5] app/testpmd: support mulitple process attach and detach port lihuisong (C) 2023-01-10 16:51 ` Ferruh Yigit 2023-01-11 0:53 ` lihuisong (C) 2023-01-11 10:27 ` Ferruh Yigit 2023-01-11 10:46 ` Ferruh Yigit 2023-01-12 2:26 ` lihuisong (C) 2023-01-18 14:12 ` Thomas Monjalon 2023-01-19 10:31 ` lihuisong (C) 2023-01-19 14:35 ` Thomas Monjalon 2023-01-28 1:39 ` lihuisong (C) 2023-01-31 3:33 ` [PATCH V5 0/5] app/testpmd: support multiple " Huisong Li 2023-01-31 3:33 ` [PATCH V5 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-01-31 3:33 ` [PATCH V5 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2023-05-22 11:04 ` fengchengwen 2023-05-27 1:58 ` lihuisong (C) 2023-01-31 3:33 ` [PATCH V5 3/5] app/testpmd: check the validity of the port Huisong Li 2023-01-31 3:33 ` [PATCH V5 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2023-01-31 3:33 ` [PATCH V5 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2023-05-16 11:27 ` [PATCH V5 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2023-05-23 0:46 ` fengchengwen 2023-05-27 2:11 ` [PATCH V6 " Huisong Li 2023-05-27 2:11 ` [PATCH V6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-05-27 2:11 ` [PATCH V6 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2023-05-27 2:11 ` [PATCH V6 3/5] app/testpmd: check the validity of the port Huisong Li 2023-05-27 2:11 ` [PATCH V6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2023-05-27 2:11 ` [PATCH V6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2023-06-06 16:26 ` [PATCH V6 0/5] app/testpmd: support multiple process attach and detach port Ferruh Yigit 2023-06-07 10:14 ` lihuisong (C) 2023-07-14 7:21 ` lihuisong (C) 2023-08-02 3:15 ` [PATCH RESEND v6 " Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 3/5] app/testpmd: check the validity of the port Huisong Li 2023-08-02 3:15 ` [PATCH RESEND v6 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2023-08-02 3:16 ` [PATCH RESEND v6 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2023-10-09 10:34 ` [PATCH RESEND v6 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2023-10-30 12:17 ` lihuisong (C) 2023-12-08 2:25 ` lihuisong (C) 2024-01-30 6:36 ` [PATCH v7 " Huisong Li 2024-01-30 6:36 ` [PATCH v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2024-01-30 6:36 ` [PATCH v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2024-01-30 6:36 ` [PATCH v7 3/5] app/testpmd: check the validity of the port Huisong Li 2024-01-30 6:36 ` [PATCH v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2024-01-30 6:36 ` [PATCH v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2024-03-08 10:38 ` [PATCH v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2024-04-23 11:17 ` lihuisong (C) 2024-09-29 5:52 ` [PATCH RESEND " Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 1/5] drivers/bus: restore driver assignment at front of probing Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 3/5] app/testpmd: check the validity of the port Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 4/5] app/testpmd: add attach and detach port for multiple process Huisong Li 2024-09-29 5:52 ` [PATCH RESEND v7 5/5] app/testpmd: stop forwarding in new or destroy event Huisong Li 2024-10-08 2:32 ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C) 2024-10-18 1:04 ` Ferruh Yigit 2024-10-18 2:48 ` lihuisong (C) 2024-10-26 4:11 ` lihuisong (C) 2024-10-29 22:12 ` Ferruh Yigit
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).