From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 91DAAA0C48; Wed, 16 Jun 2021 08:08:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 183FE410DC; Wed, 16 Jun 2021 08:08:17 +0200 (CEST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2070.outbound.protection.outlook.com [40.107.100.70]) by mails.dpdk.org (Postfix) with ESMTP id D48D140140 for ; Tue, 15 Jun 2021 13:57:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KBimEmvBlPHziv79rILb/1X85GP/7fcRqL40fqPa9GImsiwNYnb7bYAnaWLKj9JKrVLW87fSHXT7ZkaKXgYpW5ClRF67Eb7P8NE4nJSznptZtd29HpszsbaiclVD9EZcGmjayiMw6VAeBWI+dZ/n7adh9eUKmAivDwy5VCTKzxrKCV02lBkXEoS13RJNvH3b3x+G1sIPKp+sMhzBrp0Fs8wcoxQhdSWDf+TAN4Etk+Y5T4SaYxZAi23rO+NwQqDhhTz5w30CZIb4Kkw44nGNgwfYeUenGBTfYIRslfRwdqBRmQxCpPSDXQn8FcaSBQjOTg/jFE4T4R4CiZ0n9Vg/BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ijh+yZNNJi2RgC68qgU08qVWWORGw9TzAY/ouZr3AiE=; b=bT9ATARLL6ZS8PSLBSmF8N4jFLmGeyLWe7pp3+A7srrGO1XHZqPPR2K9wGGZXkYUTsSICSen1+0HlogGm7Np5RCOhsOavQJSO8wr6FQP/2UA3y9DyU6Lt3z2xfHLMoBqaXLuFcgDc49U+e3qvx56hFoO15hgB695BeJIV88qYeazZzqzOp1zDFaxetXS8Jfi9OF4ig2/KcNynRkaS/0Wu/r1S9uKx3n9dsNVk6nPWg1mfarLkeICbMiUpngQtz296TkSfsNbv97sm04Wi5Fovrsf0v6pmYSbOynIXlbc9xeEiyzMD+hk4xPy1n4tPvSNLbOpQJo8cWOpHQ+x1HSlEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ijh+yZNNJi2RgC68qgU08qVWWORGw9TzAY/ouZr3AiE=; b=khGFE2eAFlPaijk+kEebjAk0/DSBsQiSq+UdNbhmrU+2c5w63QmmjcPamKBurlev0U9NRdmcxlp8U5/h6DI+AbvgmXXcA956nyN7H+tE14tWixf7M/G7NYPWViJWfJ+QQyYuSm7QBh2t5DjehNGN2l3nSr0MX1e93/L0LCl3GTSFGo6KKKvcWE8vgyvJf2YQA+W4G18Fct7xFNF/Ki6GhAxS8duSotk+dpfDN7A4kJVU0LAGwQwA2YLy01K9lGDFVrUmWQIbKSKw2DxpnjXJsw3lYH1Hx6UaesM/f/Nw/3V1vVulo+k5g0n+wMueaxjj5iXEWcUlk0CA0ujXk5vXwg== Authentication-Results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=nvidia.com; Received: from BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) by BL1PR12MB5205.namprd12.prod.outlook.com (2603:10b6:208:308::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.15; Tue, 15 Jun 2021 11:57:28 +0000 Received: from BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::3d51:a3b9:8611:684e]) by BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::3d51:a3b9:8611:684e%8]) with mapi id 15.20.4242.016; Tue, 15 Jun 2021 11:57:28 +0000 Date: Tue, 15 Jun 2021 08:57:27 -0300 From: Jason Gunthorpe To: Thomas Monjalon Cc: "Xia, Chenbo" , "dev@dpdk.org" , "Liang, Cunming" , "Wu, Jingjing" , "Burakov, Anatoly" , "Yigit, Ferruh" , "mdr@ashroe.eu" , "nhorman@tuxdriver.com" , "Richardson, Bruce" , "david.marchand@redhat.com" , "stephen@networkplumber.org" , "Ananyev, Konstantin" , parav@nvidia.com, xuemingl@nvidia.com Message-ID: <20210615115727.GQ1002214@nvidia.com> References: <20190715075214.16616-6-tiwei.bie@intel.com> <5205443.cqaiBGeHSM@thomas> <50744230.0ZSezZt4d8@thomas> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50744230.0ZSezZt4d8@thomas> X-Originating-IP: [47.55.113.94] X-ClientProxiedBy: MN2PR01CA0022.prod.exchangelabs.com (2603:10b6:208:10c::35) To BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (47.55.113.94) by MN2PR01CA0022.prod.exchangelabs.com (2603:10b6:208:10c::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.21 via Frontend Transport; Tue, 15 Jun 2021 11:57:28 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lt7h9-0077Gg-3P; Tue, 15 Jun 2021 08:57:27 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 463780d7-9374-4c74-947a-08d92ff4bf9f X-MS-TrafficTypeDiagnostic: BL1PR12MB5205: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e5U1nHHU9Vsgx/mrOE0m7olxNicLkebvv7CwjMviN1KM0kqSNfZy9HvGboGSDUkVnGM+b6GUZOH/6HyeGbKS+nEC8aiUt9TiMjY+KG1Ljo10mH5vfDVBZ4RQdkNMU7NNlNvVNcEUFeZMUErp0kT5vgDAN2M5ViKXHK+/xy+2fGIxh5kxbriXdNUb+vyzE3mEvEkW/bIOg2CdYMJQ7SEUD6k6d7FfYaq0blrw30W+2RJxPHVqLXRqW+oyagVVw3qybv/Y3Ik2CCycd17pHUfjAwumW8xeXCQCLNpDK3R+CO7K20Z/KO/Wq9tbrN55oo/Pn0MIYT1O/sPDBAkktsWR0SWGsNcrA342HD5rRQdhkfJeNorFxQOGtgt+HJGf7zzZncJhgI9E2S3VIirGyefarNsNpagqXAngoVPSXTRdwtyOe8y/lXMYrdXdAy0icoikfgrTk4EiPTOq4rWigFJtnjeAoeTvnMFZ1tf5XW63wS/oITvNAQQdvHiSljVgKSvW6zy5gg6i+KnKtQXNz+WsRNJa6TrmqTObnkwH46qaehF+8pDZ69AK01Eu2/fqpRwqTeW8QpjJtr+7eDCjKs/fUg6HUak8u/3mv04x9oFTwiY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB5506.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(396003)(376002)(366004)(39860400002)(2906002)(86362001)(4326008)(36756003)(186003)(1076003)(8676002)(107886003)(2616005)(6916009)(8936002)(478600001)(38100700002)(66946007)(54906003)(426003)(316002)(26005)(5660300002)(7416002)(9786002)(9746002)(66476007)(66556008)(33656002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rWm2tcrTqlFNkpbK2+MWQEnRB84O8hf3FiwqoJumVuzDK9VyapbnPa6x06+A?= =?us-ascii?Q?vwn+8hQYcGjlXb0Fu53Epec8NUrKwjMb3CgKxFRpeY6pY/iE6lze8FM9i6Nr?= =?us-ascii?Q?UBKV91jK4yT0tbRtKaJyrtisjf2yUB1ITWzZauonPCQiwaIQRp+rQ88DWgIH?= =?us-ascii?Q?8wX6nt5rsQKzWG7/gUJJEg5PStQ6HbigHkWtaKuugBPN5NosBMgxZR+OBtXf?= =?us-ascii?Q?WCJg1dIepKxAQSPf9YHOXZa+RCskR3pxcVMmgMeeFS4uBOuAHX0FhCeQV2Ys?= =?us-ascii?Q?WtpnGKWUeA/VxexylzqFLO/4FTS6RRYe6LoSNq0PtQeFYr96xhcDPYUtrgPr?= =?us-ascii?Q?WyDvlN5xyhTKKmiONmSyKf+pUeTlCk+ynrk+yN9hs2rDz5h7YgOdEbhaaefa?= =?us-ascii?Q?1XUzLknjpjnxulSXQ/F1Gk2KdH1R855glu8a5ZFL/Luti+bM/Nbc6kLsxkfF?= =?us-ascii?Q?JzXKhGsxjveBKNkXQ8mVZdeUb51tKMjFMucGJ/hsi4VmHR3YibnPK4n9lnGo?= =?us-ascii?Q?l+Kn9SgZeyr3oTN4TXdGtUZ6i5ZhVmBAUSIxyI0nwbCy1XOEEcgm8cNemuS8?= =?us-ascii?Q?4g9TZhDcNygW9OCqo52+R1BdZK1ZGWP1qeSF5u97TOhQB21gg5YPhHCsreW0?= =?us-ascii?Q?YdZ/PsPNpbrPS+3ZaRIXG8MviA+Er/DfqeV5GIBnraE4FFJOhDtarUGGGoSr?= =?us-ascii?Q?mWgdQ7vJozxPa2rrRkeaq/Qk4LjHAharHrkWdqqsMwamqJgfed9sIzPiEor7?= =?us-ascii?Q?jWzhzA+4KVM/GG8F6zm29bxQ3eAWcqvvltUQllFJVPGle+QkcBEJNZqzH3Zo?= =?us-ascii?Q?X1Q82dQacmbR4T6gb0f3eegILD+5VocJel1rmLz3MamQ5v8PH0i7y4KOiej0?= =?us-ascii?Q?I4E+2d5GrGNpE2LYTIAqMPP77T/qc2W6kpEYMOUxbrfJ2AEwa6aqgkWorpRs?= =?us-ascii?Q?pC031pRqIs6ZcLKUrSE14VPu9im/2s3jJphmPP2ZNzDrYWhMeSP0OVxEnyzs?= =?us-ascii?Q?iML34me3k86ASxDBqmhQk1ISbsocquxzbIi2SCTiMb5PMepIPphgpZXc0KMQ?= =?us-ascii?Q?NXL6+UeRycHzipRs7SWfqh//X7yIpNHCDvI4QaIoRbY+SBKt70dF4f+Xh28B?= =?us-ascii?Q?pbuC+U/vIbWj7LL4Yl99l6WETJliquczFg5iu37J43bMtEWlgHVT1e7aLl+v?= =?us-ascii?Q?JQ67uct8QiTYqIAkKaGq4kLJX+6vhHaFT4lxPyKmqSV+3eEet2Kn5DEzg/hC?= =?us-ascii?Q?wqZiOu4KkdVTTjIF7gA3+iyrfSKX859YeuT0w7rRbpqb0tfYg3cph6JFUJXq?= =?us-ascii?Q?qy+GKXamYr4nKp9oCbsD9hn6?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 463780d7-9374-4c74-947a-08d92ff4bf9f X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2021 11:57:28.5965 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Qw3TrNGIAxhAFYYyEm9rEwYP0SQNDLDaOB/TF02yL56eZS/ZHQA7keKuXxaOjiVS X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5205 X-Mailman-Approved-At: Wed, 16 Jun 2021 08:08:14 +0200 Subject: Re: [dpdk-dev] [RFC v3 0/6] Add mdev (Mediated device) support in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Jun 15, 2021 at 09:48:24AM +0200, Thomas Monjalon wrote: > 15/06/2021 04:49, Xia, Chenbo: > > From: Thomas Monjalon > > > 01/06/2021 05:06, Chenbo Xia: > > > > Hi everyone, > > > > > > > > This is a draft implementation of the mdev (Mediated device [1]) > > > > support in DPDK PCI bus driver. Mdev is a way to virtualize devices > > > > in Linux kernel. Based on the device-api (mdev_type/device_api), > > > > there could be different types of mdev devices (e.g. vfio-pci). > > > > > > Please could you illustrate with an usage of mdev in DPDK? > > > What does it enable which is not possible today? > > > > The main purpose is for DPDK to drive mdev-based devices, which is not > > possible today. > > > > I'd take PCI devices for an example. Currently DPDK can only drive devices > > of physical pci bus under /sys/bus/pci and kernel exposes the pci devices > > to APP in that way. > > > > But there are PCI devices using vfio-mdev as a software framework to expose > > Mdev to APP under /sys/bus/mdev. Devices could choose this way of virtualizing > > itself to let multiple APPs share one physical device. For example, Intel > > Scalable IOV technology is known to use vfio-mdev as SW framework for Scalable > > IOV enabled devices (and Intel net/crypto/raw devices support this tech). For > > those mdev-based devices, DPDK needs support on the bus layer to scan/plug/probe/.. > > them, which is the main effort this patchset does. There are also other devices > > using the vfio-mdev framework, AFAIK, Nvidia's GPU is the first one using mdev > > and Intel's GPU virtualization also uses it. > > Yes mdev was designed for virtualization I think. > The use of mdev for Scalable IOV without virtualization > may be seen as an abuse by Linux maintainers, > as they currently seem to prefer the auxiliary bus (which is a real bus). > > Mellanox got a push back when trying to use mdev for the same purpose > (Scalable Function, also called Sub-Function) in the kernel. > The Linux community decided to use the auxiliary bus. > > Any other feedback on the choice mdev vs aux? > Is there any kernel code supporting this mdev model for Intel devices? IMHO until a kernel networking driver is accepted that uses mdev this is all just dead code in dpdk and shouldn't be merged. I think it is unlikely that future networking drivers will use mdev. > > > > 2: A new mdev bus that scans mediated pci devices and probes mdev driver to > > > > plug-in pci devices to pci bus And we are likely not doing 'mediated pci devices' at all.. Jason