Re: [dpdk-dev] [PATCH v2 0/9] Introduce vfio-user library

["Xia, Chenbo" <chenbo.xia@intel.com>]

Hi David,

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Friday, January 15, 2021 3:58 PM
> To: Xia, Chenbo <chenbo.xia@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Stephen
> Hemminger <stephen@networkplumber.org>; Liang, Cunming
> <cunming.liang@intel.com>; Lu, Xiuchun <xiuchun.lu@intel.com>; Li, Miao
> <miao.li@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Subject: Re: [PATCH v2 0/9] Introduce vfio-user library
> 
> Hello Chenbo,
> 
> On Thu, Jan 14, 2021 at 7:19 AM Chenbo Xia <chenbo.xia@intel.com> wrote:
> >
> > This series enables DPDK to be an alternative I/O device emulation library
> of
> > building virtualized devices in separate processes outside QEMU. It
> introduces
> > a new library for device emulation (librte_vfio_user).
> >
> > *librte_vfio_user* library is an implementation of VFIO-over-socket[1] (also
> > known as vfio-user) which is a protocol that allows a device to be
> virtualized
> > in a separate process outside of QEMU.
> >
> > Background & Motivation
> > -----------------------
> > The disaggregated/multi-process QEMU is using VFIO-over-socket/vfio-user
> > as the main transport mechanism to disaggregate IO services from QEMU[2].
> > Vfio-user essentially implements the VFIO device model presented to the
> > user process by a set of messages over a unix-domain socket. The main
> > difference between application using vfio-user and application using vfio
> > kernel module is that device manipulation is based on socket messages for
> > vfio-user but system calls for vfio kernel module. The vfio-user devices
> > consist of a generic VFIO device type, living in QEMU, which is called the
> > client[3], and the core device implementation (emulated device), living
> > outside of QEMU, which is called the server. With emulated devices removed
> > from QEMU enabled by vfio-user implementation, other places should be
> > introduced to accommodate virtualized/emulated device. This series
> introduces
> > vfio-user support in DPDK to enable DPDK as one of the living places for
> > emulated device except QEMU.
> >
> > This series introduce the server and client implementation of vfio-user
> protocol.
> > The server plays the role as emulated devices and the client is the device
> > consumer. With this implementation, DPDK will be enabled to be both device
> > provider and consumer.
> >
> > Design overview
> > ---------------
> >
> > +--------------+     +--------------+
> > | +----------+ |     | +----------+ |
> > | | Generic  | |     | | Emulated | |
> > | | vfio-dev | |     | | device   | |
> > | +----------+ |     | +----|-----+ |
> > | +----------+ |     | +----|-----+ |
> > | | vfio-user| |     | | vfio-user| |
> > | | client   | |<--->| | server   | |
> > | +----------+ |     | +----------+ |
> > | QEMU/DPDK    |     | DPDK         |
> > +--------------+     +--------------+
> >
> > - Generic vfio-dev.
> >   It is the generic vfio framework in vfio applications like QEMU or DPDK.
> >   Applications can keep the most of vfio device management and plug in a
> >   vfio-user device type. Note that in current implementation, we have not
> >   yet integrated client vfio-user into kernel vfio in DPDK but it is viable
> >   and good to do so.
> >
> > - vfio-user client.
> >   For DPDK, it is part of librte_vfio_user implementation to provide ways to
> >   manipulate a vfio-user based emulated devices. This manipulation is very
> >   similar with kernel vfio (i.e., syscalls like ioctl, mmap and
> pread/pwrite).
> >   It is a base for vfio-user device consumer.
> >
> > - vfio-user server.
> >   It is server part of librte_vfio_user. It provides ways to emulate your
> own
> >   device. A device provider could only care about device layout that VFIO
> >   defines but does not need to know how it communicates with vfio-user
> client.
> >
> > - Emulated device.
> >   It is emulated device of any type (e.g., network, crypto and etc.).
> >
> > References
> > ----------
> > [1]: https://patchew.org/QEMU/20201130161229.23164-1-
> thanos.makatos@nutanix.com/
> > [2]: https://wiki.qemu.org/Features/MultiProcessQEMU
> > [3]: https://github.com/oracle/qemu/tree/vfio-user-v0.2
> >
> > ----------------------------------
> > v2:
> >  - Clean up non-static inline function (Stephen)
> >  - Naturally pack vfio-user message payload and header (Stephen)
> >  - Make function definiton align with coding style (Beilei)
> >  - Clean up duplicate code in vfio-user server APIs (Beilei)
> >  - Fix some typos
> 
> 
> GHA (called by the robot) caught various issues:
> - doc:
> https://github.com/ovsrobot/dpdk/runs/1700373705?check_suite_focus=true#step:1
> 4:3407
> - clang:
> https://github.com/ovsrobot/dpdk/runs/1700373722?check_suite_focus=true#step:1
> 4:1050
> - i386:
> https://github.com/ovsrobot/dpdk/runs/1700373747?check_suite_focus=true#step:1
> 4:2607
> - aarch64:
> https://github.com/ovsrobot/dpdk/runs/1700373764?check_suite_focus=true#step:1
> 4:2848
> and
> https://github.com/ovsrobot/dpdk/runs/1700373770?check_suite_focus=true#step:1
> 4:2880

Thanks! Will fix in next version.

Chenbo
> 
> 
> --
> David Marchand