Re: [dpdk-dev] [PATCH v2 0/9] Introduce vfio-user library

[David Marchand <david.marchand@redhat.com>]

Hello Chenbo,

On Thu, Jan 14, 2021 at 7:19 AM Chenbo Xia <chenbo.xia@intel.com> wrote:
>
> This series enables DPDK to be an alternative I/O device emulation library of
> building virtualized devices in separate processes outside QEMU. It introduces
> a new library for device emulation (librte_vfio_user).
>
> *librte_vfio_user* library is an implementation of VFIO-over-socket[1] (also
> known as vfio-user) which is a protocol that allows a device to be virtualized
> in a separate process outside of QEMU.
>
> Background & Motivation
> -----------------------
> The disaggregated/multi-process QEMU is using VFIO-over-socket/vfio-user
> as the main transport mechanism to disaggregate IO services from QEMU[2].
> Vfio-user essentially implements the VFIO device model presented to the
> user process by a set of messages over a unix-domain socket. The main
> difference between application using vfio-user and application using vfio
> kernel module is that device manipulation is based on socket messages for
> vfio-user but system calls for vfio kernel module. The vfio-user devices
> consist of a generic VFIO device type, living in QEMU, which is called the
> client[3], and the core device implementation (emulated device), living
> outside of QEMU, which is called the server. With emulated devices removed
> from QEMU enabled by vfio-user implementation, other places should be
> introduced to accommodate virtualized/emulated device. This series introduces
> vfio-user support in DPDK to enable DPDK as one of the living places for
> emulated device except QEMU.
>
> This series introduce the server and client implementation of vfio-user protocol.
> The server plays the role as emulated devices and the client is the device
> consumer. With this implementation, DPDK will be enabled to be both device
> provider and consumer.
>
> Design overview
> ---------------
>
> +--------------+     +--------------+
> | +----------+ |     | +----------+ |
> | | Generic  | |     | | Emulated | |
> | | vfio-dev | |     | | device   | |
> | +----------+ |     | +----|-----+ |
> | +----------+ |     | +----|-----+ |
> | | vfio-user| |     | | vfio-user| |
> | | client   | |<--->| | server   | |
> | +----------+ |     | +----------+ |
> | QEMU/DPDK    |     | DPDK         |
> +--------------+     +--------------+
>
> - Generic vfio-dev.
>   It is the generic vfio framework in vfio applications like QEMU or DPDK.
>   Applications can keep the most of vfio device management and plug in a
>   vfio-user device type. Note that in current implementation, we have not
>   yet integrated client vfio-user into kernel vfio in DPDK but it is viable
>   and good to do so.
>
> - vfio-user client.
>   For DPDK, it is part of librte_vfio_user implementation to provide ways to
>   manipulate a vfio-user based emulated devices. This manipulation is very
>   similar with kernel vfio (i.e., syscalls like ioctl, mmap and pread/pwrite).
>   It is a base for vfio-user device consumer.
>
> - vfio-user server.
>   It is server part of librte_vfio_user. It provides ways to emulate your own
>   device. A device provider could only care about device layout that VFIO
>   defines but does not need to know how it communicates with vfio-user client.
>
> - Emulated device.
>   It is emulated device of any type (e.g., network, crypto and etc.).
>
> References
> ----------
> [1]: https://patchew.org/QEMU/20201130161229.23164-1-thanos.makatos@nutanix.com/
> [2]: https://wiki.qemu.org/Features/MultiProcessQEMU
> [3]: https://github.com/oracle/qemu/tree/vfio-user-v0.2
>
> ----------------------------------
> v2:
>  - Clean up non-static inline function (Stephen)
>  - Naturally pack vfio-user message payload and header (Stephen)
>  - Make function definiton align with coding style (Beilei)
>  - Clean up duplicate code in vfio-user server APIs (Beilei)
>  - Fix some typos


GHA (called by the robot) caught various issues:
- doc: https://github.com/ovsrobot/dpdk/runs/1700373705?check_suite_focus=true#step:14:3407
- clang: https://github.com/ovsrobot/dpdk/runs/1700373722?check_suite_focus=true#step:14:1050
- i386: https://github.com/ovsrobot/dpdk/runs/1700373747?check_suite_focus=true#step:14:2607
- aarch64: https://github.com/ovsrobot/dpdk/runs/1700373764?check_suite_focus=true#step:14:2848
and https://github.com/ovsrobot/dpdk/runs/1700373770?check_suite_focus=true#step:14:2880


-- 
David Marchand