From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0E5AA0A02; Fri, 15 Jan 2021 08:58:40 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 47399140DC1; Fri, 15 Jan 2021 08:58:40 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id BF4A4140DBF for ; Fri, 15 Jan 2021 08:58:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610697518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ogWDJmIAgGvn3PL9f5+djRFjCcPULDjfVHs0AY6vQxA=; b=KfnNNTApJ+gZ5ew0bXXOQroMUlCuDKYqpjNzYY7msPSe8L3qWJ8ByGeOPCI7NUnDOg35d1 UU40Tr4vT+qLoK+7C3/reAmr/ISzzkgP6Xrp5mIaMy0J7C6dvFcXkGwETqjBGD0jSGbzSV VBZMV57QNI9mwd5pUfUTRfrvCpiR2Zc= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-349-7Zn8T2IVOlik_bpf8XAIMA-1; Fri, 15 Jan 2021 02:58:33 -0500 X-MC-Unique: 7Zn8T2IVOlik_bpf8XAIMA-1 Received: by mail-vk1-f200.google.com with SMTP id y84so3493444vkd.5 for ; Thu, 14 Jan 2021 23:58:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ogWDJmIAgGvn3PL9f5+djRFjCcPULDjfVHs0AY6vQxA=; b=gxMzVyc0FQlAXZyUqBBJHY0QawIPdaDDqFmjqDuKY7XdQMuFaecKtKB9M4h+G0dEVm XdVc6U0+xjO32dsdcNFU+NKTCYdl7x74neetORvH0Qe57SuecuMSGlwvZvC/8Jo6WtsE rz4hLccg/rkCVAiCw8Sfbcmw1162eRY7pqlH+N2VfVesXml8R4w9Hi/PF68ae7pINFcF iqAheTzluzWyHQIKjM+6r0CItGa1IbAtLdiLguJNb46KF4ByO4TkJvuBFxjFbmbfII9Z xnD9niAdB7Q4z++pe1S2yXh9qxFArnicSJheWbAGw0g59u6X7UD0zjKCMD6qD+o7ut3J Plnw== X-Gm-Message-State: AOAM531/UIiCQl9eYo7xOgbAyx5e0Hq/UM+EGl2EXvpt4Lj8lTHCuPw6 uAD8YertGFr10LonEcsJfr55WDpxMCIjH4Ls218G+HyGunx2u5SAL95n7J26nK1tHpA0IcXY2Q+ Yw+xAMXEBpKNMkmgyo5E= X-Received: by 2002:a67:87d1:: with SMTP id j200mr10084984vsd.10.1610697512990; Thu, 14 Jan 2021 23:58:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJx7SAJOyFR9/Mh3RPRWplG3BMyFPIRxD7jVyTK+2BZJph2IetR+ZgPyvvQbsrqfrM4jWmxgbqpiS4MwbFo7g0E= X-Received: by 2002:a67:87d1:: with SMTP id j200mr10084968vsd.10.1610697512681; Thu, 14 Jan 2021 23:58:32 -0800 (PST) MIME-Version: 1.0 References: <20201218073851.93609-1-chenbo.xia@intel.com> <20210114061411.39166-1-chenbo.xia@intel.com> In-Reply-To: <20210114061411.39166-1-chenbo.xia@intel.com> From: David Marchand Date: Fri, 15 Jan 2021 08:58:21 +0100 Message-ID: To: Chenbo Xia Cc: dev , Thomas Monjalon , Stephen Hemminger , "Liang, Cunming" , xiuchun.lu@intel.com, miao.li@intel.com, Jingjing Wu , Beilei Xing Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2 0/9] Introduce vfio-user library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hello Chenbo, On Thu, Jan 14, 2021 at 7:19 AM Chenbo Xia wrote: > > This series enables DPDK to be an alternative I/O device emulation library of > building virtualized devices in separate processes outside QEMU. It introduces > a new library for device emulation (librte_vfio_user). > > *librte_vfio_user* library is an implementation of VFIO-over-socket[1] (also > known as vfio-user) which is a protocol that allows a device to be virtualized > in a separate process outside of QEMU. > > Background & Motivation > ----------------------- > The disaggregated/multi-process QEMU is using VFIO-over-socket/vfio-user > as the main transport mechanism to disaggregate IO services from QEMU[2]. > Vfio-user essentially implements the VFIO device model presented to the > user process by a set of messages over a unix-domain socket. The main > difference between application using vfio-user and application using vfio > kernel module is that device manipulation is based on socket messages for > vfio-user but system calls for vfio kernel module. The vfio-user devices > consist of a generic VFIO device type, living in QEMU, which is called the > client[3], and the core device implementation (emulated device), living > outside of QEMU, which is called the server. With emulated devices removed > from QEMU enabled by vfio-user implementation, other places should be > introduced to accommodate virtualized/emulated device. This series introduces > vfio-user support in DPDK to enable DPDK as one of the living places for > emulated device except QEMU. > > This series introduce the server and client implementation of vfio-user protocol. > The server plays the role as emulated devices and the client is the device > consumer. With this implementation, DPDK will be enabled to be both device > provider and consumer. > > Design overview > --------------- > > +--------------+ +--------------+ > | +----------+ | | +----------+ | > | | Generic | | | | Emulated | | > | | vfio-dev | | | | device | | > | +----------+ | | +----|-----+ | > | +----------+ | | +----|-----+ | > | | vfio-user| | | | vfio-user| | > | | client | |<--->| | server | | > | +----------+ | | +----------+ | > | QEMU/DPDK | | DPDK | > +--------------+ +--------------+ > > - Generic vfio-dev. > It is the generic vfio framework in vfio applications like QEMU or DPDK. > Applications can keep the most of vfio device management and plug in a > vfio-user device type. Note that in current implementation, we have not > yet integrated client vfio-user into kernel vfio in DPDK but it is viable > and good to do so. > > - vfio-user client. > For DPDK, it is part of librte_vfio_user implementation to provide ways to > manipulate a vfio-user based emulated devices. This manipulation is very > similar with kernel vfio (i.e., syscalls like ioctl, mmap and pread/pwrite). > It is a base for vfio-user device consumer. > > - vfio-user server. > It is server part of librte_vfio_user. It provides ways to emulate your own > device. A device provider could only care about device layout that VFIO > defines but does not need to know how it communicates with vfio-user client. > > - Emulated device. > It is emulated device of any type (e.g., network, crypto and etc.). > > References > ---------- > [1]: https://patchew.org/QEMU/20201130161229.23164-1-thanos.makatos@nutanix.com/ > [2]: https://wiki.qemu.org/Features/MultiProcessQEMU > [3]: https://github.com/oracle/qemu/tree/vfio-user-v0.2 > > ---------------------------------- > v2: > - Clean up non-static inline function (Stephen) > - Naturally pack vfio-user message payload and header (Stephen) > - Make function definiton align with coding style (Beilei) > - Clean up duplicate code in vfio-user server APIs (Beilei) > - Fix some typos GHA (called by the robot) caught various issues: - doc: https://github.com/ovsrobot/dpdk/runs/1700373705?check_suite_focus=true#step:14:3407 - clang: https://github.com/ovsrobot/dpdk/runs/1700373722?check_suite_focus=true#step:14:1050 - i386: https://github.com/ovsrobot/dpdk/runs/1700373747?check_suite_focus=true#step:14:2607 - aarch64: https://github.com/ovsrobot/dpdk/runs/1700373764?check_suite_focus=true#step:14:2848 and https://github.com/ovsrobot/dpdk/runs/1700373770?check_suite_focus=true#step:14:2880 -- David Marchand