Re: [External] : Re: Query Regarding Race Condition Between Packet Reception and Device Stop in DPDK

["Van Haaren, Harry" <harry.van.haaren@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 3142 bytes --]

Side note; please "reply inline" in plaint text when sending to mailing lists, it allows future readers to see the context of your question when reading the reply.

> From: Changchun Zhang <changchun.zhang@oracle.com>
> Sent: Monday, February 17, 2025 7:14 PM
> To: Stephen Hemminger <stephen@networkplumber.org>
> Cc: Van Haaren, Harry <harry.van.haaren@intel.com>; NAGENDRA BALAGANI <nagendra.balagani@oracle.com>; users@dpdk.org <users@dpdk.org>
> Subject: Re: [External] : Re: Query Regarding Race Condition Between Packet Reception and Device Stop in DPDK
>
> Okay, so here the issue is still rte_eth_dev_stop(), but not rte_eth_dev_rx_queue_stop(), right? I mean, as long as not calling rte_eth_dev_stop() on control path, is it safe to call rte_eth_dev_rx_queue_stop/rte_eth_dev_rx_queue_start on control path while fast path keeps calling rte_eth_rx_burst()?

I see Stephen has already directly answered your question - but perhaps the below is interesting to expand beyond the exact question being asked.

Instead of answering specific questions like above, allow me to explain the "mental model" around how I think about ports/queues and threads using them.
- Each Queue (rx or tx, doesn't matter) is an entity that can be polled by a dataplane thread. No control plane actions (e.g. start/stop/reset) may be active if the queue is polled!
- Each Queue depends on a port (that it logically belongs to). That port must be started for the queue to be usable from the dataplane thread.
- Each Queue is an individual object: different dataplane threads can poll different queues without any "interference" (allowing one queue to restart, while another STAYS in use)

For specific use-cases, you can now logic out what is required:
- Stop Queue: the dataplane thread MUST NOT poll if another thread is operating on the the same queue.
- Stop Port: All queues must be stopped before the port can be stopped. Therefore, all dataplane threads must stop using the queues associated with the port.

The DPDK docs e.g. rte_eth_dev_start: https://doc.dpdk.org/api/rte__ethdev_8h.html#afdc834c1c52e9fb512301990468ca7c2 do have some statements around expected usage: "On success, all basic functions exported by the Ethernet API (link status, receive/transmit, and so on) can be invoked." Perhaps the documentation can be improved - if you're willing to contribute, this would likely be appreciated by the next developer learning the correct usage of these APIs?

As folks on list may know, I've been using the Rust language for a number of years, and it has some very nice properties around encoding these types of details at the API layer, and putting compile-time (or runtime - depends on implementation) enforcement on these API constraints. This was presented at DPDK Userspace '23, with the Rust + DPDK Ethdev part of the talk being relevant to the above "mental model": this timestamp (6 minutes in) is the start of that section: https://youtu.be/lb6xn2xQ-NQ?t=359

> Thanks,
> Changchun

Regards! -Harry

<snip previous discussions; see https://mails.dpdk.org/archives/users/2025-February/thread.html#8122>

[-- Attachment #2: Type: text/html, Size: 8475 bytes --]