From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5BD3D42940; Fri, 14 Apr 2023 12:49:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2D92340144; Fri, 14 Apr 2023 12:49:07 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2073.outbound.protection.outlook.com [40.107.92.73]) by mails.dpdk.org (Postfix) with ESMTP id 2C05D400D5 for ; Fri, 14 Apr 2023 12:49:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hl0nC4+z+2qMn1+IJZxAxr5hbVZUrOLBmKv0lONDtGYh68Z3fuf0NBlkHuz8I4SRp8yKOxwf+PF7s8X1GeXQRrjcAlaxEzZuTQhP2kZq+DHyzC7R/QG5JtAOjpOAinWnTWnekatqcUy/PeXSzZrnhp0u+2AEzeDuJXjeUiozajCCV7mr+7+x2ADtW9lsmegY+q2GizyQVp7zT0EBnGI5f4bLY5Y/9xlvZnhcphFcZRZ6gGepw+IWv2B5hIH859D8cVUMCJPhUO6ZFHxfM6f2x/Coogg/TzSY8xEWCABgyMTnji/m3jzeHiw5Gb+/J5sq+RDvPzbkRLQcHksceiGC+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HTBrqMR4iZxzcSEvmKHb+I/PVP1rWDEj2u/oaPlJnSo=; b=iURRUCCvwYIaZEUO4c3T+GkPqQNBTZ0OGbu9tmUWRQHud6RCpRT+MPlG6rCnQDq4SC9ZHaXMZuSHyvI4PS+I8rZ9TKwy2A2BjK7SiH5FtAHDBAxKxL4AyeJazUjw9taFEKqDIYjueC3u4TC9HodL6iHxpeGXBj5vl4m1z+A/BAIxwaBaPku7YfLx9d0Eok4NVsLLIfJ3Kiyhxwqe3stRmLFdjeAfIqscugUfn5+ip4dNcqwm5tgC/fZgFQzC6JPEc2dy9H8Uc7+2CIj6oLLHbLXhYSwqkiozZ8cLSIFUCehqEToIfNdhIwlp86RGapdF3++V3AuXob/PaT2js1J7Jg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HTBrqMR4iZxzcSEvmKHb+I/PVP1rWDEj2u/oaPlJnSo=; b=2glG0ibljlApODbUSD97uHLNJOP5LqmXXD835MzqV5nSuQJgS8ajJ/uCJDPgvIEsM/4s4lFKX54VzOzf7QjsVChRvYzC24DHoL+X/IhFC1XdXLdJ9FFQNMsy8pUJ+LBu4yde40i9GCjaBDbENK6/hvtq/CdpTaKGYIUl03jvqFs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by PH7PR12MB5688.namprd12.prod.outlook.com (2603:10b6:510:130::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.30; Fri, 14 Apr 2023 10:49:03 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::5e2c:c0ed:88a6:a4c7]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::5e2c:c0ed:88a6:a4c7%7]) with mapi id 15.20.6298.030; Fri, 14 Apr 2023 10:49:03 +0000 Message-ID: Date: Fri, 14 Apr 2023 11:48:55 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: Maxime Coquelin , "Xia, Chenbo" , =?UTF-8?Q?Morten_Br=c3=b8rup?= , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" References: <20230331154259.1447831-1-maxime.coquelin@redhat.com> <3789c0c7-281b-89c3-45aa-3d985736b04a@amd.com> <107f53d8-c0c3-68a8-1857-4b6ef0165b48@redhat.com> <98CBD80474FA8B44BF855DF32C47DC35D8786B@smartserver.smartshare.dk> <58480498-8037-ec45-548a-8027e185fcaf@redhat.com> From: Ferruh Yigit Subject: Re: [RFC 00/27] Add VDUSE support to Vhost library In-Reply-To: <58480498-8037-ec45-548a-8027e185fcaf@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P265CA0379.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::31) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|PH7PR12MB5688:EE_ X-MS-Office365-Filtering-Correlation-Id: aed40d92-3ab6-4370-7282-08db3cd5dc9c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9x4i9BHakb1WsXyHozhFixnLaimdDJ7lJUWiYlsS2tGD7loPW8PU9RDUeQNZ3e8h+G8m0kg/MYMJv7e3RBHOyXHC5A1NBpKpAUawgP7gm/l7hH3UqHQ/42Tpiw7ppmFkMkKAAoZejoT41OXucHqwfNxR6h8uyUw3OlX3lTFW8ciWH+5kveC1awgQzQO3XVZFBJcvhXyZz2iN3mNDaMKGx1YopN6h9CQZFuMXR17ljntve23HxYxL3h4RDxV9Lu+WS2SmwEh94t9WF3JU8TIJVSyzzqh20jrNyCIBzYOl6CFWtr2IWZRGAuKdN/xVH/NPQIXiqHJxHuIxr1DkmHoCGAFhXMJeVLnE2oYSi7oJvVkX7mTh/whaU3qG81khrNwJjyITe4Nw125TjmDgYNkuBSL7irhd5vUVL36T49Yi9iytB8W2R0qalHPBPeMILZyn1+uvDZ7khqlLTjp+KTypXGbFerS5ELx0r+HIJ08/mCZ/yYl+Yn8Kht2xxLNj3+JMHHAYDOvx/m7+iJ5+Y5sNchBgPpHCj451187/+CfAJUUhDh7/SC8WaGn6rW3zA0ZKCoXw+GV6AeJkQiq71Z1Qb6yNTQ7W9FIQ9uUyxIy9zEVXxTtSkX10ZPfCPV3c2C5Gc5cBnC1S1AAb8n6VXN2XNA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(136003)(396003)(366004)(39860400002)(451199021)(8676002)(7416002)(44832011)(2616005)(5660300002)(36756003)(8936002)(186003)(41300700001)(38100700002)(2906002)(83380400001)(921005)(66574015)(316002)(110136005)(478600001)(53546011)(6512007)(31686004)(6506007)(26005)(86362001)(66476007)(66556008)(66946007)(6666004)(31696002)(6486002)(966005)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VzRuc2MzVUlNa2xLMk1JRjI4SGdhRGhEdDJBRTFnSGNsL0VHWnJqTmhJdDNa?= =?utf-8?B?R2NlVFNvY2orZm5rQWZlZlYvU0kzM3YrWkNZakE5OEYyWmRWUFQ1dlpVSTND?= =?utf-8?B?d3E2ZUt5S1FiOEJnVmVaZTlMd2FGcG9wa3ViSHRpdnViNWxkTktsZTE5TG9r?= =?utf-8?B?N21rU1pNUVp3NGxLbGNlc2F5aHpyTzkvVFdwQWdhNmFWWjdBb2g5OXdZbG5Z?= =?utf-8?B?dmJaNTRpL0pFUzl1UEp0RHBiS1ZjKzljLzlTTno0NWFpQjdySlA0TldXNjlj?= =?utf-8?B?MWdtdWw4MnNOZlc4M2JSRXBuOVFNL0dqSGZ3QjlFMU9BdjNqSk05K3lETGNU?= =?utf-8?B?OXVuRlVvTndWL1RKRklpSmdLR0VCU29OMksxMmw2NzZDM204bjFLdGw0OW9W?= =?utf-8?B?c2I5ZnJBd0tkYXhuTmhGRUlZeVlXejFib3ZmdGptKzMvY1ptRkhNYUkwaVBZ?= =?utf-8?B?R3NBeUcwT2QzUkpJQVdmM0p6REdldDBlNmpnNnhNSTZsdVJWajcyWS85SEFw?= =?utf-8?B?WUM0MEE4Tm5tLzk1M21JUnM5Q2U4NldtNmFVSG5JQUVGU1NMYTRyQUI4aGhF?= =?utf-8?B?VW5HM0hPU2RkaVJPdUtBYmdVbUZSWEtlMEVkYkZGaWt6LzlhL2lRZmZsTFFS?= =?utf-8?B?cWZnNktGMlFTRW84YW95TTBaNHZLVS9lZFRYM0I5L2U3cXpTV0NWKzA5VXBM?= =?utf-8?B?Wm5jRjhNUE91YmVUc2IrcHNPYnViNUJMaUpPdzRxck4rV1YwMHZmckRWdUZs?= =?utf-8?B?WFVUMFYvZTNJWk5ZNEM2ZzhXN0hGZ2FQazV5R2ZiaGRnZGQ4WXgzSy9UR01X?= =?utf-8?B?ZEhxcXJ1ZEhLMDZvaE5hdGozN0VFRVdpOHhCdXBWVHRaMGZ5ZG5ML2dSRzhL?= =?utf-8?B?RVpudTZVQmlhZkttRTdqZHpQSFpWQmdYZU1Kb1M4aWRUR08rSnJuYXpzZHp4?= =?utf-8?B?YUQ1WHhHTW51VDBhclg5NExPNjV5ckFMdURNdUltbGpyWXBNV3VyM0dGYU5R?= =?utf-8?B?ajBsWXNvRjdvWk1sL2Jla3pSQVFsbGF2bXhNMHExdWx3ajhjMzc1NUNaSUEz?= =?utf-8?B?ZURQSVcxUGZyd1Q3N2NESmtoWHNLREdFR3EwWEtxRGlGRVR0MEFEVlZSczIw?= =?utf-8?B?R2syNG5qUnRMcXlLdGxzVUllVE9FUTJpQmt2cVJHNDUvam1BSm16VngrTGp1?= =?utf-8?B?SzdsSWhaNHF1ZkJPVkd3MkZDMitxVDJwY0oxK21xb1pQWWpmS3hTUnY0ak9M?= =?utf-8?B?TWlZQmg4NGFaclhCaytHaGtUcmpVWVIydWRVUnJSWDc3d2pNdytScmpxMjhu?= =?utf-8?B?ZTZyUXhBWkdiY3RLemMxWjRXRkF5ekE3OS9zeHpiNElEMEhBWkd4RFhOTk9W?= =?utf-8?B?aHlFL3o5R2d5eDRUYkxyR0NKRTJVS3VvZlpsekJLWjVDT1NweS9vTzBWbEVC?= =?utf-8?B?eHBLY3NYQm9EV0VXTGZYM3FnejNCZG1IVkIxcDRoL2p4V3B0MjcwVEZUWWM2?= =?utf-8?B?YThBN2ppTml3MnBHRm1uSG9KTzFnQ1AyYm5md0JxdFpUSGVJVFE5cFdWRWRX?= =?utf-8?B?L3ZQdWVxTFpSeGpmVzVCVlVYZ3JUb09sNmdhQ2gxQVo5dkJuZTByZDZvcW5w?= =?utf-8?B?aE5tOEFQUEVsTjR4RWY1QTB6ZDVVNXJEaUtENWJ2VVF1TUYrNGxFd3JkcGNW?= =?utf-8?B?R2JUTjk3KzRaQjI2UCtFODVlV2VmQnFNNnY4YmxDNW8wYTY1UkMxek1yc1Bh?= =?utf-8?B?cW5NY2dmd1lkTXNwdTNWQndaY3FNU3lFV3pieDJqWUo4UjJQaVY2K0NJM1RR?= =?utf-8?B?bzd4Y0ZGVHRnVUlENE0xdlMwek5hTGVxeVByWFdaK1hMaEdnb2JMcEdpUXc0?= =?utf-8?B?TENnS0Vlb09xaEs3Rlg2YjFHMkdLMHl5a2FIWDI3d1MwaTQyaW1MeW9BMEJC?= =?utf-8?B?YVBRRWRIRHFkMVNRMytMY0ZIUHdhVzgyaWRtL0NZY0xRNDdaT2dvazlPSWpU?= =?utf-8?B?VmM2aEsyeU5SOXhUaGwzQzlNQU9iUi93cDZKaWtwWVUrK1I5anJ5eFdOL1dC?= =?utf-8?B?dnI5Y2ptVi91R1pDQS90SFJiS0hVWmpCNzJCdXpSdU5BSGhqSmxGZUJMOGRG?= =?utf-8?Q?xgrCxu9TDvJqrOYfQYj2oRjFa?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aed40d92-3ab6-4370-7282-08db3cd5dc9c X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2023 10:49:03.3015 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: O1genurj4mRi12XQDSlLtAdVXkggLTajiAeOBckBQZB1EVUuDqi2K49DGQ36r2G7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5688 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 4/13/2023 8:59 AM, Maxime Coquelin wrote: > Hi, > > On 4/13/23 09:08, Xia, Chenbo wrote: >>> -----Original Message----- >>> From: Morten Brørup >>> Sent: Thursday, April 13, 2023 3:41 AM >>> To: Maxime Coquelin ; Ferruh Yigit >>> ; dev@dpdk.org; david.marchand@redhat.com; Xia, >>> Chenbo ; mkp@redhat.com; fbl@redhat.com; >>> jasowang@redhat.com; Liang, Cunming ; Xie, >>> Yongji >>> ; echaudro@redhat.com; eperezma@redhat.com; >>> amorenoz@redhat.com >>> Subject: RE: [RFC 00/27] Add VDUSE support to Vhost library >>> >>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>> Sent: Wednesday, 12 April 2023 17.28 >>>> >>>> Hi Ferruh, >>>> >>>> On 4/12/23 13:33, Ferruh Yigit wrote: >>>>> On 3/31/2023 4:42 PM, Maxime Coquelin wrote: >>>>>> This series introduces a new type of backend, VDUSE, >>>>>> to the Vhost library. >>>>>> >>>>>> VDUSE stands for vDPA device in Userspace, it enables >>>>>> implementing a Virtio device in userspace and have it >>>>>> attached to the Kernel vDPA bus. >>>>>> >>>>>> Once attached to the vDPA bus, it can be used either by >>>>>> Kernel Virtio drivers, like virtio-net in our case, via >>>>>> the virtio-vdpa driver. Doing that, the device is visible >>>>>> to the Kernel networking stack and is exposed to userspace >>>>>> as a regular netdev. >>>>>> >>>>>> It can also be exposed to userspace thanks to the >>>>>> vhost-vdpa driver, via a vhost-vdpa chardev that can be >>>>>> passed to QEMU or Virtio-user PMD. >>>>>> >>>>>> While VDUSE support is already available in upstream >>>>>> Kernel, a couple of patches are required to support >>>>>> network device type: >>>>>> >>>>>> https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_poc >>>>>> >>>>>> In order to attach the created VDUSE device to the vDPA >>>>>> bus, a recent iproute2 version containing the vdpa tool is >>>>>> required. >>>>> >>>>> Hi Maxime, >>>>> >>>>> Is this a replacement to the existing DPDK vDPA framework? What is the >>>>> plan for long term? >>>>> >>>> >>>> No, this is not a replacement for DPDK vDPA framework. >>>> >>>> We (Red Hat) don't have plans to support DPDK vDPA framework in our >>>> products, but there are still contribution to DPDK vDPA by several vDPA >>>> hardware vendors (Intel, Nvidia, Xilinx), so I don't think it is going >>>> to be deprecated soon. >>> >>> Ferruh's question made me curious... >>> >>> I don't know anything about VDUSE or vDPA, and don't use any of it, so >>> consider me ignorant in this area. >>> >>> Is VDUSE an alternative to the existing DPDK vDPA framework? What are >>> the >>> differences, e.g. in which cases would an application developer (or >>> user) >>> choose one or the other? >> >> Maxime should give better explanation.. but let me just explain a bit. >> >> Vendors have vDPA HW that support vDPA framework (most likely in their >> DPU/IPU >> products). This work is introducing a way to emulate a SW vDPA device in >> userspace (DPDK), and this SW vDPA device also supports vDPA framework. >> >> So it's not an alternative to existing DPDK vDPA framework :) > > Correct. > > When using DPDK vDPA, the datapath of a Vhost-user port is offloaded to > a compatible physical NIC (i.e. a NIC that implements Virtio rings > support), the control path remains the same as a regular Vhost-user > port, i.e. it provides a Vhost-user unix socket to the application (like > QEMU or DPDK Virtio-user PMD). > > When using Kernel vDPA, the datapath is also offloaded to a vDPA > compatible device, and the control path is managed by the vDPA bus. > It can either be consumed by a Kernel Virtio device (here Virtio-net) > when using Virtio-vDPA. In this case the device is exposed as a regular > netdev and, in the case of Kubernetes, can be used as primary interfaces > for the pods. > Or it can be exposed to user-space via Vhost-vDPA, a chardev that can be > seen as an alternative to Vhost-user sockets. In this case it can for > example be used by QEMU or DPDK Virtio-user PMD. In Kubernetes, it can > be used as a secondary interface. > > Now comes VDUSE. VDUSE is a Kernel vDPA device, but instead of being a > physical device where the Virtio datapath is offloaded, the Virtio > datapath is offloaded to a user-space application. With this series, a > DPDK application, like OVS-DPDK for instance, can create VDUSE device > and expose them either as regular netdev when binding them to Kernel > Virtio-net driver via Virtio-vDPA, or as Vhost-vDPA interface to be > consumed by another userspace appliation like QEMU or DPDK application > using Virtio-user PMD. With this solution, OVS-DPDK could serve both > primary and secondary interfaces of Kubernetes pods. > > I hope it clarifies, I will add these information in the cover-letter > for next revisions. Let me know if anything is still unclear. > > I did a presentation at last DPDK summit [0], maybe the diagrams will > help to clarify furthermore. > Thanks Chenbo, Maxime for clarification. After reading a little more (I think) I got it better, slides [0] were useful. So this is more like a backend/handler, similar to vhost-user, although it is vDPA device emulation. Can you please describe more the benefit of vduse comparing to vhost-user? Also what is "VDUSE daemon", which is referred a few times in documentation, is it another userspace implementation of the vduse? >>> >>> And if it is a better alternative, perhaps the documentation should >>> mention that it is recommended over DPDK vDPA. Just like we started >>> recommending alternatives to the KNI driver, so we could phase it out >>> and >>> eventually get rid of it. >>> >>>> >>>> Regards, >>>> Maxime >> > > [0]: > https://static.sched.com/hosted_files/dpdkuserspace22/9f/Open%20DPDK%20to%20containers%20networking%20with%20VDUSE.pdf >