Re: [PATCH] rust: RFC/demo of safe API for Dpdk Eal, Eth and Rxq

["Etelson, Gregory" <getelson@nvidia.com>]

Hello Harry,

Thank you for sharing the API.
Please check out my comments below.

Regards,
Gregory

On Thu, 17 Apr 2025, Harry van Haaren wrote:

> External email: Use caution opening links or attachments
>
>
> This patch is NOT to be considered for merge, it is a demo
> of the Rust APIs for Ethdev. There is no actual implementation
> of the APIs against the DPDK C functions, this is Rust API only.
>
> To test/run the code (and uncomment things to see errors)
> just apply this patch, cd "rust_api_example" and run
> $ cargo run
>
> This will compile the API, and spawn 2x threads to poll on
> two Rxq instances. The comments in the code explain how the
> "Send" and "Sync" attributes are captured per instances of a
> struct (e.g. how RxqHandle -> Rxq restricts thread movement).
>
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---
> rust_api_example/Cargo.toml  |   6 ++
> rust_api_example/src/main.rs | 189 +++++++++++++++++++++++++++++++++++
> 2 files changed, 195 insertions(+)
> create mode 100644 rust_api_example/Cargo.toml
> create mode 100644 rust_api_example/src/main.rs
>
> diff --git a/rust_api_example/Cargo.toml b/rust_api_example/Cargo.toml
> new file mode 100644
> index 0000000000..0137826340
> --- /dev/null
> +++ b/rust_api_example/Cargo.toml
> @@ -0,0 +1,6 @@
> +[package]
> +name = "rust_api_example"
> +version = "0.1.0"
> +edition = "2021"
> +
> +[dependencies]
> diff --git a/rust_api_example/src/main.rs b/rust_api_example/src/main.rs
> new file mode 100644
> index 0000000000..8d0de50c30
> --- /dev/null
> +++ b/rust_api_example/src/main.rs
> @@ -0,0 +1,189 @@
> +// Outline for safe DPDK API bindings
> +//  - None of the APIs are actually implemented, this is API design only
> +//  - This demo runs 2x threads on 2x Rxqs, and cannot accidentally poll incorrectly
> +
> +pub mod dpdk {
> +    pub mod eth {
> +        use super::Mempool;
> +
> +        #[derive(Debug)]
> +        pub struct TxqHandle {/* todo: but same as Rxq */}
> +
> +        // Handle allows moving between threads, its not polling!
> +        #[derive(Debug)]
> +        pub struct RxqHandle {
> +            port: u16,
> +            queue: u16,
> +        }
> +
> +        impl RxqHandle {
> +            pub(crate) fn new(port: u16, queue: u16) -> Self {
> +                RxqHandle { port, queue }
> +            }
> +
> +            // This function is the key to the API design: it ensures the rx_burst()
> +            // function is only available via the Rxq struct, after enable_polling() has been called.
> +            // It "consumes" (takes "self" as a parameter, not a '&' reference!) which essentially
> +            // destroys/invalidates the handle from the Application level code.
> +
> +            // It returns an Rxq instance, which has the PhantomData to encode the threading requirements,
> +            // and the Rxq has the rx_burst() function: this allows the application to recieve packets.
> +            pub fn enable_polling(self) -> Rxq {
> +                Rxq {
> +                    handle: self,
> +                    _phantom: std::marker::PhantomData,
> +                }
> +            }
> +        }
> +
> +        #[derive(Debug)]
> +        pub struct Rxq {
> +            handle: RxqHandle,
> +            // This "PhantomData" tells the rust compiler to Pretend the Rc<()> is in this struct
> +            // but in practice it is a Zero-Sized-Type, so takes up no space. It is a compile-time
> +            // language technique to ensure the struct is not moved between threads. This encodes
> +            // the API requirement "don't poll from multiple threads without synchronisation (e.g. Mutex)"
> +            _phantom: std::marker::PhantomData<std::rc::Rc<()>>,
> +        }
> +
> +        impl Rxq {
> +            // TODO: datapath Error types should be lightweight, not String. Here we return ().
> +            pub fn rx_burst(&mut self, _mbufs: &mut [u8]) -> Result<usize, ()> {
> +                // TODO: Design the Mbuf struct wrapper, and how to best return a batch
> +                //  e.g.: investigate "ArrayVec" crate for safe & fixed sized, stack allocated arrays
> +                //
> +                // There is work to do here, but I want to communicate the general DPDK/EAL/Eth/Rxq concepts
> +                // now, this part is not done yet: it is likely the hardest/most performance critical.
> +                //
> +                // call rte_eth_rx_burst() here
> +                println!(
> +                    "[thread: {:?}] rx_burst: port {} queue {}",
> +                    std::thread::current().id(),
> +                    self.handle.port,
> +                    self.handle.queue
> +                );
> +                Ok(0)
> +            }
> +        }
> +
> +        #[derive(Debug)]
> +        pub struct Port {
> +            id: u16,
> +            rxqs: Vec<RxqHandle>,
> +            txqs: Vec<TxqHandle>,
> +        }
> +
> +        impl Port {
> +            // pub(crate) here ensures outside this crate users cannot call this function
> +            pub(crate) fn from_u16(id: u16) -> Self {
> +                Port {
> +                    id,
> +                    rxqs: Vec::new(),
> +                    txqs: Vec::new(),
> +                }
> +            }
> +
> +            pub fn rxqs(&mut self, rxq_count: u16, _mempool: Mempool) -> Result<(), String> {
> +                for q in 0..rxq_count {
> +                    // call rte_eth_rx_queue_setup() here
> +                    self.rxqs.push(RxqHandle::new(self.id, q));
> +                }
> +                Ok(())
> +            }
> +
> +            pub fn start(&mut self) -> (Vec<RxqHandle>, Vec<TxqHandle>) {
> +                // call rte_eth_dev_start() here, then give ownership of Rxq/Txq to app

After a call to Port::start, Rx and Tx queues are detached from it's port.
With that model how rte_eth_dev_stop() and subsequent rte_eth_dev_start()
DPDK calls can be implemented ?

> +                (
> +                    std::mem::take(&mut self.rxqs),
> +                    std::mem::take(&mut self.txqs),
> +                )
> +            }
> +        }
> +    }
> +
> +    #[derive(Debug, Clone)]
> +    // Mempool is a long-life object, which many other DPDK things refer to (e.g. rxq config)
> +    // Having a Rust lifetime attached to it (while technically correct) would complicate the
> +    // code a LOT, and for little value. This is a tradeoff - happy to discuss more if we want.
> +    // The choice here is to derive "Clone", allowing handing over multiple instances of the
> +    // same Mempool, similar to how Arc<Mempool> would work, but without the reference counting.
> +    pub struct Mempool {}
> +
> +    impl Mempool {
> +        pub fn new(_size: usize) -> Self {
> +            Self {}
> +        }
> +    }
> +
> +    #[derive(Debug)]
> +    pub struct Eal {
> +        eth_ports: Option<Vec<eth::Port>>,
> +    }
> +
> +    impl Eal {
> +        //  allow init once,
> +        pub fn init() -> Result<Self, String> {
> +            // EAL init() will do PCI probe and VDev enumeration will find/create eth ports.
> +            // This code should loop over the ports, and build up Rust structs representing them
> +            let eth_port = vec![eth::Port::from_u16(0)];
> +            Ok(Eal {
> +                eth_ports: Some(eth_port),
> +            })
> +        }
> +
> +        // API to get eth ports, taking ownership. It can be called once.
> +        // The return will be None for future calls
> +        pub fn take_eth_ports(&mut self) -> Option<Vec<eth::Port>> {
> +            self.eth_ports.take()
> +        }
> +    }
> +
> +    impl Drop for Eal {
> +        fn drop(&mut self) {
> +            // todo: rte_eal_cleanup()
> +        }
> +    }
> +} // DPDK mod
> +
> +fn main() {
> +    let mut dpdk = dpdk::Eal::init().expect("dpdk must init ok");
> +    let rx_mempool = dpdk::Mempool::new(4096);
> +
> +    let mut ports = dpdk.take_eth_ports().expect("take eth ports ok");

Eal::take_eth_ports() resets EAL ports.
A call to rte_dev_probe() will ether fail, because Eal::eth_ports is None 
or create another port-0, depending on implementation.

> +    let mut p = ports.pop().unwrap();
> +
> +    p.rxqs(2, rx_mempool).expect("rxqs setup ok");
> +    println!("{:?}", p);
> +
> +    let (mut rxqs, _txqs) = p.start();
> +    println!("rxqs: {:?}", rxqs);
> +
> +    let rxq1 = rxqs.pop().unwrap();
> +    let rxq2 = rxqs.pop().unwrap();
> +
> +    // spawn a new thread to use rxq1. This demonstrates that the RxqHandle
> +    // type can move between threads - it is not tied to the thread that created it.
> +    std::thread::spawn(move || {
> +        // Uncomment this: it fails to compile!
> +        //   - Rxq2 would be used by this newly-spawned thread
> +        //     -- specifically the variable was "moved" into this thread
> +        //   - it is also used below (by the main thread)
> +        // "value used after move" is the error, on the below code
> +        // let mut rxq = rxq2.enable_polling();
> +
> +        // see docs on enable_polling above to understand how the enable_polling()
> +        // function helps to achieve the thread-safety-at-compile-time goal.
> +        let mut rxq = rxq1.enable_polling();
> +        loop {
> +            let _nb_mbufs = rxq.rx_burst(&mut [0; 32]);
> +            std::thread::sleep(std::time::Duration::from_millis(1000));
> +        }
> +    });
> +
> +    // main thread polling rxq2
> +    let mut rxq = rxq2.enable_polling();
> +    loop {
> +        let _nb_mbufs = rxq.rx_burst(&mut [0; 32]);
> +        std::thread::sleep(std::time::Duration::from_millis(1000));
> +    }
> +}
> --
> 2.34.1
>
>