From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 83C65A034C; Tue, 29 Mar 2022 19:46:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 722B040691; Tue, 29 Mar 2022 19:46:20 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 84AD840141 for ; Tue, 29 Mar 2022 19:46:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648575978; x=1680111978; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=sV8SvX67lTIuKr/cD/zt6MIY+yAwBdaqpmWQX57jLfk=; b=BUrNCb49LGn4hKYK1tqrzKmuLr4X69GtjqR3IQbUpeH7MOQQDKvw1sXU szmjEpDVXsYlDxO44uEmCKWbOcDckVq5xaGRHc9W/aDtr7ibmSPdZ+LJ6 1lVJbCoJl8yyHnATuWJuw69jBmZyEM5Gz0lwz7eep8RoMIbWSG1tibQqL MvbyGAJ2cD+elK6J0AqUdSFM3yLHmEH8lCSNGONUlm8oFxAR5YE+7W8Vx bK6VSvrVnmGnNv8L4+njUV/jCf8HB/P1moz12CpRUQvzZU/pbIqUgMPdn FnkzHZbgKGKK7P9ZFxuEkJosYlbu61sWkLhNGml9dZjyIJ6oG6OvYX/rf g==; X-IronPort-AV: E=McAfee;i="6200,9189,10301"; a="345758540" X-IronPort-AV: E=Sophos;i="5.90,220,1643702400"; d="scan'208";a="345758540" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Mar 2022 10:46:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,220,1643702400"; d="scan'208";a="653143694" Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85]) by orsmga004.jf.intel.com with ESMTP; 29 Mar 2022 10:46:17 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 29 Mar 2022 10:46:17 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Tue, 29 Mar 2022 10:46:17 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.21; Tue, 29 Mar 2022 10:46:16 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=njvPIH367JVaq8xROgd7XEkfOzjm+g0trxL8kcPrV4kSG6BbBZSuj95m1jCTzwyN+TB3dn05N3bwIQ4pON5p/mlviXZ0SnQx6g7uWnYS/b0yPzBom0fMr8UkXvZCnDrcXF+IvMHa7DKLBr6q5wsM0PB3musEWkutz9e1zfRWm54yaPTGtmql9WZCxS6Tc5a6QHp0Au2me1j49YEM/p+mZawCwPe9QzZeheHDxVYDtDVNULtu40mah59m/ppNa/YHv8LlzdPxabvmFeUUU5KIZrzxELh3k6Zp4LDgaAm67qQWDYPirnqMuD6anJivXyQBnRvD8eqqi6ByjKUWS2aVCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qp75yYGhYqPsJG7CKrOHpG7FwyJ0rWHZ8Q4yhbMVlv8=; b=ixL3VAz97r8kuL42kzmJthgSJCfdqeiACcX+3NxlpYl03MMlD/tmLJbfooAtz9N25yHfG6+mMQVJXkCCyD3MJXY3zBdoUfpocJVmtvjAN9PU0zHjbEq8MXrPFYV6X+vp0OIYMIZubIC44dQYlN1rc/SfCz+2Hly7aaRTkVkmXXbzg5MLifYjb3M2fbdlvhO+HuHWOD9n175+iYpB8bScXTjLbiKb1NfN6TUlVcs28nf9RowEB2oCBBghpX/bNpsuA/XNJaK4zuSreeyW7WLXkMzCokXCJpVMLcf0r8xTMR5F3Wc3kXSk4uY/tK2Hmu0fF0gSl76M0h+UJm4jNoTB0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN0PR11MB5712.namprd11.prod.outlook.com (2603:10b6:408:160::17) by DM6PR11MB3708.namprd11.prod.outlook.com (2603:10b6:5:146::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.16; Tue, 29 Mar 2022 17:46:13 +0000 Received: from BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::28cf:55af:8c4b:d4d9]) by BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::28cf:55af:8c4b:d4d9%3]) with mapi id 15.20.5102.023; Tue, 29 Mar 2022 17:46:13 +0000 From: "Van Haaren, Harry" To: =?iso-8859-1?Q?Morten_Br=F8rup?= , "Richardson, Bruce" CC: Maxime Coquelin , "Pai G, Sunil" , "Stokes, Ian" , "Hu, Jiayu" , "Ferriter, Cian" , "Ilya Maximets" , "ovs-dev@openvswitch.org" , "dev@dpdk.org" , "Mcnamara, John" , "O'Driscoll, Tim" , "Finn, Emma" Subject: RE: OVS DPDK DMA-Dev library/Design Discussion Thread-Topic: OVS DPDK DMA-Dev library/Design Discussion Thread-Index: Adg/jDNGcC8G4wWtSxeVfUOuAS3Y6wACLuFQAM6falAAJoZBIAAA0srQAAK2F/AABHBTAAAAvWSAAACgo4AAAF2UgAAAVPwg Date: Tue, 29 Mar 2022 17:46:13 +0000 Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D86F7C@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D86F7D@smartserver.smartshare.dk> <7968dd0b-8647-8d7b-786f-dc876bcbf3f0@redhat.com> <98CBD80474FA8B44BF855DF32C47DC35D86F7E@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D86F80@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86F80@smartserver.smartshare.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.401.20 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: edf37639-c8e1-4e46-cc9c-08da11ac04c9 x-ms-traffictypediagnostic: DM6PR11MB3708:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: V8RzZWCAD880aHsgeUA1W0CXaJlXI8alOzmImS5gQ51Y3DNrsX4pQjhc6XPMnz+7MwzMEwudnADj3jS+m7GVsLCo+csTcEk+lYjfW4uxKeZ8YazBYBz2qA1K14gD1bW21hK1Pyi0QMAqr9JpmkawFuN2L3p9D4tHKFhSL6opFJNAxXNpi4OqXD/+bMsNByBfbjGeO6BUfWFmt4OHAGEzmbC++SoHUFy9zQDhuv46DT7SRTI+VYOIC9M1GH3UwfRzJIJZW/Ex304UwogmJD9lUKGGNBdFGPSaFQR1OEE5TrvqS/ft1D05ONk5vppCW+NSp4Cvvy3WEENtqYQesk5QDn3M3mCNfgug9j2b6eR5hyS1sYPvHoubB1d+OBeeQUK+xJbdTUz7h58efDjkC85wHgF8qTAO3erfOIYcWT4zNF8yTaKOrVl58Wj8fLBmt7S9Z9JwCdu5BwHpm5PQWGRYhFlj27FCi5jZJxHveQPafGWAdM/Kk9q2zsMZ6ihgWKXUzSbBUYwFnWr7wYOjhZ8H1rgQV8K9+wGRhCgqoOAFaTgavOlDUju4UYKUdXCkXX2cSBmmdgXGJ3pqD3rv9Pz9nYfVjqcJFG67K8d4WOlpBhHOsoQPOno/e+QNyyJZOdOYA3Clb6ko4v77KSOo9mvGWRZscAPZNqwM3mZANuPYxL7r+k7KhXUEslhRs1PMlIGqpGGixgml3h1W6COFWO8IEQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN0PR11MB5712.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6506007)(7696005)(9686003)(110136005)(53546011)(54906003)(71200400001)(508600001)(33656002)(4326008)(66946007)(8676002)(52536014)(76116006)(66446008)(8936002)(64756008)(5660300002)(66476007)(66556008)(6636002)(86362001)(316002)(2906002)(38070700005)(186003)(26005)(38100700002)(82960400001)(83380400001)(55016003)(107886003)(66574015)(122000001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?Nq5br1FwnudugAap7hIkmDKranpsgAGkhrfSIuMiWM7WhfTwOImPf9A2P/?= =?iso-8859-1?Q?VC9+8G18EublRMYUvg/H8ZUuSnAlG0dF1bO+8yYOQJzaV4EWhMJLdrbMXW?= =?iso-8859-1?Q?qDa0sh/Tj8c3HoWHXIiw296gaRTZ7Fn1sNmJRx0fiPPzHDtLZJk7IOXcGu?= =?iso-8859-1?Q?qADXhbTuxYYbKPucP29MCXZZdpeD0DOeLnFxF74cUquAed3znWxdaP6QIc?= =?iso-8859-1?Q?aV3w4V3JACS1+x3Fft1iFIVBXRAFBgFM0Byls4+x7lHsGEg6o5PxnJnUgf?= =?iso-8859-1?Q?YT/vl+xGl2pg06Ysiovjyzc7gyWHAX9D/r7X8cgMRRvZss+jZ81nZNdFiX?= =?iso-8859-1?Q?S1uJuul/ocSX9krfW91gmW+0dBC4bq7s1oSJZTE7lDQAMBrTOd6r/GgRpY?= =?iso-8859-1?Q?CN7NP5XCMDPFFGkonogTbmcj1i7uyc+ze0BL4LCucuaK/gDdBxrqeneq7S?= =?iso-8859-1?Q?Q1VOK+RQ2JLQgb9GERxtikVSd84mvvBL+m5Gogh6GIG4Ki2UOl9VUg2lxu?= =?iso-8859-1?Q?U/OcvOiVtbUffamAcDmNGW4cQSapV5koOe24oCx8EJ9K+l5CKC27TnguLN?= =?iso-8859-1?Q?31U+caJ128LQ3CjYeFeRpKO89pMQfYOrae1erGeVTR/WBiMmJmLKpCN31f?= =?iso-8859-1?Q?Ci6gALFAdl6l71apZZMqglqmhhRO+lbyEjn9URlefvDjZJP1HFR2ShroqM?= =?iso-8859-1?Q?z25cK2+XGSXD38WP1iwbtgab6Vr+thiBd/rNpkgS72e/WvhieIER4n+M6h?= =?iso-8859-1?Q?/qx7ebEpuN199BAH6CU6TwIcbq2dKmcQ7iRKBUhC4G0wSNDMjGuHLTsWA6?= =?iso-8859-1?Q?NZJpbSun+Iin/R+lOWmCaF5HmOj2z8CgH/FeAcdkZAuR1gjbRLifWl5+Fk?= =?iso-8859-1?Q?PLb48T7FPOoOg9NoC6YI3CvsDBtVaoYpkOfcvfe+hhaQg7qrQXcafXYK0Z?= =?iso-8859-1?Q?4SHaljL9ji0neuB/KARfPRAHxKHbEdKSu4rYUuDcqRP8UrQgQ+VNA0pV2p?= =?iso-8859-1?Q?10cXbc8IKYIELhbHc4qPwa2qzxlaZWWwwTcG4teQKjphYRv4AoHAgi91v9?= =?iso-8859-1?Q?aj/4BVxMEiqxNctwKgtJcacZalq778znlXZaQmWHmmP0SMAHaHn4osLefB?= =?iso-8859-1?Q?1g0cZfRaj7qQ5TAWk7K0GpMkiiHemOa9b3WtdS8yTufczOLOWFeBd8Shiw?= =?iso-8859-1?Q?o1+dQhwZc7WEnKXVzk1QAZsKLmmY7YDucv+rDGT56Ong74Agke4g6DcXSA?= =?iso-8859-1?Q?yG46hsqo7CesErbG4w++18FOMRkQjNhix/eY7cXQ41E0jRMK0RhGiPhXye?= =?iso-8859-1?Q?trjhPaQiKYl504Vr46cp1jKrqSYdcEWzwGkx87A4+fEvYRBfpuGQ+f0HoD?= =?iso-8859-1?Q?k2fc6IgC395iVnAcewsd1BVrhU0YVXa1i5ZXI+SDAS8K+IsGqdmVNn0rJ4?= =?iso-8859-1?Q?YIhdY0WKMlh/lj3zGn0o/jeucvr5Lft+4w66j6BJQqt0TLTfqNgkomOVZ7?= =?iso-8859-1?Q?Ayx055EzZB12FgaeuYeQB2pXzlqMfQkvPrzIO8VWCj6zYcoaP125kShLgL?= =?iso-8859-1?Q?dIPuTvypPouAqFF+tjzr3PMLLLiGvG9FzW9OTxxogW2MCHnE8ciegCLTJO?= =?iso-8859-1?Q?bTQ0PnlVHZo+YofAaJzGjZFOPE3prA/51NW95xyRZcmgxi98YzVXHXiOEP?= =?iso-8859-1?Q?f4R4+2N7dvPkugk+3iTHgG+x82UBOvfBLGxnfXRBzR8uMn+WEW3calZptB?= =?iso-8859-1?Q?WkAaoliogPYBXxnzn75mMe69S31bSyP3E5tKMjKZhXnsvlKmIJGID8GPsr?= =?iso-8859-1?Q?/zTwyYtih5GUT5QNrRCPT8NPVP6cd70=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5712.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: edf37639-c8e1-4e46-cc9c-08da11ac04c9 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Mar 2022 17:46:13.6540 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ajHOOMuTKflKIOJVekAbZUxKrZxDWGQWZdqN8/NkeQD5JBs+WUCaUiaSub+2t5wusGzW5ptZW2jJfAoEoYDT4ZL/VJl13M+4fN0yRyRSEfk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB3708 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Morten Br=F8rup > Sent: Tuesday, March 29, 2022 6:14 PM > To: Richardson, Bruce > Cc: Maxime Coquelin ; Van Haaren, Harry > ; Pai G, Sunil ; Stoke= s, Ian > ; Hu, Jiayu ; Ferriter, Cian > ; Ilya Maximets ; ovs- > dev@openvswitch.org; dev@dpdk.org; Mcnamara, John > ; O'Driscoll, Tim ; Fin= n, > Emma > Subject: RE: OVS DPDK DMA-Dev library/Design Discussion >=20 > > From: Bruce Richardson [mailto:bruce.richardson@intel.com] > > Sent: Tuesday, 29 March 2022 19.03 > > > > On Tue, Mar 29, 2022 at 06:45:19PM +0200, Morten Br=F8rup wrote: > > > > From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] > > > > Sent: Tuesday, 29 March 2022 18.24 > > > > > > > > Hi Morten, > > > > > > > > On 3/29/22 16:44, Morten Br=F8rup wrote: > > > > >> From: Van Haaren, Harry [mailto:harry.van.haaren@intel.com] > > > > >> Sent: Tuesday, 29 March 2022 15.02 > > > > >> > > > > >>> From: Morten Br=F8rup > > > > >>> Sent: Tuesday, March 29, 2022 1:51 PM > > > > >>> > > > > >>> Having thought more about it, I think that a completely > > different > > > > architectural approach is required: > > > > >>> > > > > >>> Many of the DPDK Ethernet PMDs implement a variety of RX and TX > > > > packet burst functions, each optimized for different CPU vector > > > > instruction sets. The availability of a DMA engine should be > > treated > > > > the same way. So I suggest that PMDs copying packet contents, e.g. > > > > memif, pcap, vmxnet3, should implement DMA optimized RX and TX > > packet > > > > burst functions. > > > > >>> > > > > >>> Similarly for the DPDK vhost library. > > > > >>> > > > > >>> In such an architecture, it would be the application's job to > > > > allocate DMA channels and assign them to the specific PMDs that > > should > > > > use them. But the actual use of the DMA channels would move down > > below > > > > the application and into the DPDK PMDs and libraries. > > > > >>> > > > > >>> > > > > >>> Med venlig hilsen / Kind regards, > > > > >>> -Morten Br=F8rup > > > > >> > > > > >> Hi Morten, > > > > >> > > > > >> That's *exactly* how this architecture is designed & > > implemented. > > > > >> 1. The DMA configuration and initialization is up to the > > application > > > > (OVS). > > > > >> 2. The VHost library is passed the DMA-dev ID, and its new > > async > > > > rx/tx APIs, and uses the DMA device to accelerate the copy. > > > > >> > > > > >> Looking forward to talking on the call that just started. > > Regards, - > > > > Harry > > > > >> > > > > > > > > > > OK, thanks - as I said on the call, I haven't looked at the > > patches. > > > > > > > > > > Then, I suppose that the TX completions can be handled in the TX > > > > function, and the RX completions can be handled in the RX function, > > > > just like the Ethdev PMDs handle packet descriptors: > > > > > > > > > > TX_Burst(tx_packet_array): > > > > > 1. Clean up descriptors processed by the NIC chip. --> Process > > TX > > > > DMA channel completions. (Effectively, the 2nd pipeline stage.) > > > > > 2. Pass on the tx_packet_array to the NIC chip descriptors. -- > > > Pass > > > > on the tx_packet_array to the TX DMA channel. (Effectively, the 1st > > > > pipeline stage.) > > > > > > > > The problem is Tx function might not be called again, so enqueued > > > > packets in 2. may never be completed from a Virtio point of view. > > IOW, > > > > the packets will be copied to the Virtio descriptors buffers, but > > the > > > > descriptors will not be made available to the Virtio driver. > > > > > > In that case, the application needs to call TX_Burst() periodically > > with an empty array, for completion purposes. This is what the "defer work" does at the OVS thread-level, but instead of "brute-forcing" and *always* making the call, the defer work concept tracks *when* there is outstanding work (DMA copies) to be completed ("deferred wo= rk") and calls the generic completion function at that point. So "defer work" is generic infrastructure at the OVS thread level to handle work that needs to be done "later", e.g. DMA completion handling. > > > Or some sort of TX_Keepalive() function can be added to the DPDK > > library, to handle DMA completion. It might even handle multiple DMA > > channels, if convenient - and if possible without locking or other > > weird complexity. That's exactly how it is done, the VHost library has a new API added, which= allows for handling completions. And in the "Netdev layer" (~OVS ethdev abstractio= n) we add a function to allow the OVS thread to do those completions in a new Netdev-abstraction API called "async_process" where the completions can be = checked. The only method to abstract them is to "hide" them somewhere that will alwa= ys be polled, e.g. an ethdev port's RX function. Both V3 and V4 approaches use t= his method. This allows "completions" to be transparent to the app, at the tradeoff to = having bad separation of concerns as Rx and Tx are now tied-together.=20 The point is, the Application layer must *somehow * handle of completions. So fundamentally there are 2 options for the Application level: A) Make the application periodically call a "handle completions" function A1) Defer work, call when needed, and track "needed" at app layer, and cal= ling into vhost txq complete as required. Elegant in that "no work" means "no cycles spent" on checking DMA = completions. A2) Brute-force-always-call, and pay some overhead when not required. Cycle-cost in "no work" scenarios. Depending on # of vhost queues,= this adds up as polling required *per vhost txq*. Also note that "checking DMA completions" means taking a virtq-loc= k, so this "brute-force" can needlessly increase x-thread contention! B) Hide completions and live with the complexity/architectural sacrifice of= mixed-RxTx.=20 Various downsides here in my opinion, see the slide deck presented earlier= today for a summary.=20 In my opinion, A1 is the most elegant solution, as it has a clean separatio= n of concerns, does not cause avoidable contention on virtq locks, and spends no cycles when there is no = completion work to do. > > > Here is another idea, inspired by a presentation at one of the DPDK > > Userspace conferences. It may be wishful thinking, though: > > > > > > Add an additional transaction to each DMA burst; a special > > transaction containing the memory write operation that makes the > > descriptors available to the Virtio driver. > > > > > > > That is something that can work, so long as the receiver is operating > > in > > polling mode. For cases where virtio interrupts are enabled, you still > > need > > to do a write to the eventfd in the kernel in vhost to signal the > > virtio > > side. That's not something that can be offloaded to a DMA engine, > > sadly, so > > we still need some form of completion call. >=20 > I guess that virtio interrupts is the most widely deployed scenario, so l= et's ignore > the DMA TX completion transaction for now - and call it a possible future > optimization for specific use cases. So it seems that some form of comple= tion call > is unavoidable. Agree to leave this aside, there is in theory a potential optimization, but unlikely to be of large value.