Re: DPDK for File Transfer and Writing

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 11 Jun 2025 08:08:20 +0000 (UTC)
Isaac Lim <izlim@yahoo.com> wrote:

> Dear Stephen,
> Thank you so much.
> Regarding SPDX, I did a Google search. Is it Software Package Data Exchange?
> Best Regards,Isaac 
> 
>     On Tuesday, June 10, 2025 at 09:36:45 PM GMT+8, Stephen Hemminger <stephen@networkplumber.org> wrote:  
>  
>  On Tue, 10 Jun 2025 08:49:02 +0000 (UTC)
> Isaac Lim <izlim@yahoo.com> wrote:
> 
> > Dear all,
> > We are using DPDK to transfer files and write to SSD on the receiver side.
> > On the sender side, we are splitting the files into 1024-byte packets before sending over to the receiver.
> > Currently, on the sender side, we are using a worker lcore to retrieve the packets and storing the file data in a buffer while the main lcore writes the data to file using write().
> > However, we are realizing that the write speed is not fast enough to keep up with the number of packets being received.
> > We have tried using io_uring to aid speeding up the writing but it seems to be even slower than just writing the file normally.
> > We would like to check with the community if you have any DPDK compatible suggestions that would help improve file transfer and write performance?
> > Thank you very much in advance.
> > Best Regards,Isaac LIM  
> 
> You could try SPDX to get faster storage, but networking is faster than storage.
> 
> At some point you need a full network stack with real protocols like TCP.
>   

SPDK https://spdk.io/