From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 63425A0C40; Fri, 11 Jun 2021 09:15:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DDDFC40688; Fri, 11 Jun 2021 09:15:38 +0200 (CEST) Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) by mails.dpdk.org (Postfix) with ESMTP id C17C74014F for <dev@dpdk.org>; Fri, 11 Jun 2021 09:15:37 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.nyi.internal (Postfix) with ESMTP id 062E9580822; Fri, 11 Jun 2021 03:15:36 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 11 Jun 2021 03:15:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm1; bh= RWMg3ff/DDEIKCR0gcGlkoLg8GsM39gUD4j2Vu/fZjk=; b=525nNUwjNm09p/br DFrsWsge6YJkMoi1Mf6lAuO0FXApWmzVTCGC6A2zF7RnqCVY55Wmna3nywY1iN2U cIwdUam5CjvqpelyKoGEYmGnWW7CTPQHQ+aIyni6NFMcTkIwV7bxwFGCzSrMYaBx 1n6J/Mxb6/lIEHCk9xYG0IkPjVSGKt470zzYN5Ux9Ys0CBDRdGKXjHHShyNsG/ow +MNcXCMUSh5zhSNsZNR4tkM3whnZk+iFm5s7eq8iUef+245k2fiHaL62+0IJ4fhR Z78+YVVmcG9ecHL9+I53e4OsmgOqkDMBCMk3BUELm+YjzPucgKKZcrGsfsoyXp+y 8d8b2g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=RWMg3ff/DDEIKCR0gcGlkoLg8GsM39gUD4j2Vu/fZ jk=; b=TUUBP+mdzqnXJL8/EkBPxEO28CdVhcI6bGH7g7W/hCKl+u573ioUHJpnw w1C8JBK6N9QsFgTDSqi2UzV9D+GyOKI3srZG84yJIEJL9CEM83a1vsvCkAlWZzsR jPrU99ygEseXIGbx5OkjR77u4AL0/ApL+B7FZ7szXgb8ItqbJOxzBFRZYzUpdZeS hXH+a0/2J852/c4sqMkK28F3GuK3nPlfoRI1OhM6mUgbLKIvSU2Zw+VU5rQBiBP2 ZnPDHfgRyCTE3MOkcRDIPm2wGfcMQbmFwjyV/uWLJJbnGrYe7QAhBzR7wo7GRFzW boALI1KPwrIShGbpwoEWj7ttADI9w== X-ME-Sender: <xms:lg3DYDnvNL8UVXDYy7mD6jJs6_xItDf0spFRV_gA_kPj8JSKwsDtWQ> <xme:lg3DYG0t-Tber9qWWl7YDfsCPhKCVmtc3Al65-zOpL1EW52lanDjuA-nuJJr304y7 qHqKFXF09RRqXJUIQ> X-ME-Received: <xmr:lg3DYJqdxKQ2_yLMwxTPEi7JX_7SdPd_1lePTxDyJF4wJQiFWvjoD9aQsmVssymhe3Oj1aTJykKEF9jHo1KVPOkGVg> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeduiedgudduiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpeffvdffjeeuteelfeeileduudeugfetjeelveefkeejfeeigeeh teffvdekfeegudenucffohhmrghinhepughpughkrdhorhhgnecuvehluhhsthgvrhfuih iivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhho nhdrnhgvth X-ME-Proxy: <xmx:lg3DYLniKug_Bzcdv_0N29G_wPbJce4VUnmyjA5DdUtdY1Lmnna4xw> <xmx:lg3DYB2DxA7Gc0GAuTfsDhQwyi0g0RS8gbVY3NtGCKrGAu84uyKSVw> <xmx:lg3DYKt9Gl2VSlWOXA1zyN8J_HuaFxfhKcmJS23LCx5GusgiuNKBZA> <xmx:mA3DYKsWdiJjGIsYFBPKepb7q1k7_ltk4JIsxvYoPjAGIvuBs68O8A> Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 11 Jun 2021 03:15:33 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: Chenbo Xia <chenbo.xia@intel.com> Cc: dev@dpdk.org, cunming.liang@intel.com, jingjing.wu@intel.com, anatoly.burakov@intel.com, ferruh.yigit@intel.com, mdr@ashroe.eu, nhorman@tuxdriver.com, bruce.richardson@intel.com, david.marchand@redhat.com, stephen@networkplumber.org, konstantin.ananyev@intel.com Date: Fri, 11 Jun 2021 09:15:32 +0200 Message-ID: <5205443.cqaiBGeHSM@thomas> In-Reply-To: <20210601030644.3318-1-chenbo.xia@intel.com> References: <20190715075214.16616-6-tiwei.bie@intel.com> <20210601030644.3318-1-chenbo.xia@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [RFC v3 0/6] Add mdev (Mediated device) support in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> 01/06/2021 05:06, Chenbo Xia: > Hi everyone, > > This is a draft implementation of the mdev (Mediated device [1]) > support in DPDK PCI bus driver. Mdev is a way to virtualize devices > in Linux kernel. Based on the device-api (mdev_type/device_api), > there could be different types of mdev devices (e.g. vfio-pci). Please could you illustrate with an usage of mdev in DPDK? What does it enable which is not possible today? > In this patchset, the PCI bus driver is extended to support scanning > and probing the mdev devices whose device-api is "vfio-pci". > > +---------+ > | PCI bus | > +----+----+ > | > +--------+-------+-------+--------+ > | | | | > Physical PCI devices ... Mediated PCI devices ... > > The first four patches in this patchset are mainly preparation of mdev > bus support. The left two patches are the key implementation of mdev bus. > > The implementation of mdev bus in DPDK has several options: > > 1: Embed mdev bus in current pci bus > > This patchset takes this option for an example. Mdev has several > device types: pci/platform/amba/ccw/ap. DPDK currently only cares > pci devices in all mdev device types so we could embed the mdev bus > into current pci bus. Then pci bus with mdev support will scan/plug/ > unplug/.. not only normal pci devices but also mediated pci devices. I think it is a different bus. It would be cleaner to not touch the PCI bus. Having a separate bus will allow an easy way to identify a device with the new generic devargs syntax, example: bus=mdev,uuid=XXX or more complex: bus=mdev,uuid=XXX/class=crypto/driver=qat,foo=bar > 2: A new mdev bus that scans mediated pci devices and probes mdev driver to > plug-in pci devices to pci bus > > If we took this option, a new mdev bus will be implemented to scan > mediated pci devices and a new mdev driver for pci devices will be > implemented in pci bus to plug-in mediated pci devices to pci bus. > > Our RFC v1 takes this option: > http://patchwork.dpdk.org/project/dpdk/cover/20190403071844.21126-1-tiwei.bie@intel.com/ > > Note that: for either option 1 or 2, device drivers do not know the > implementation difference but only use structs/functions exposed by > pci bus. Mediated pci devices are different from normal pci devices > on: 1. Mediated pci devices use UUID as address but normal ones use BDF. > 2. Mediated pci devices may have some capabilities that normal pci > devices do not have. For example, mediated pci devices could have > regions that have sparse mmap capability, which allows a region to have > multiple mmap areas. Another example is mediated pci devices may have > regions/part of regions not mmaped but need to access them. Above > difference will change the current ABI (i.e., struct rte_pci_device). > Please check 5th and 6th patch for details. > > 3. A brand new mdev bus that does everything > > This option will implement a new and standalone mdev bus. This option > does not need any changes in current pci bus but only needs some shared > code (linux vfio part) in pci bus. Drivers of devices that support mdev > will register itself as a mdev driver and do not rely on pci bus anymore. > This option, IMHO, will make the code clean. The only potential problem > may be code duplication, which could be solved by making code of linux > vfio part of pci bus common and shared. Yes I prefer this third option. We can find an elegant way of sharing some VFIO code between buses.