From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 30E1AA0505; Thu, 7 Apr 2022 18:05:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD2CD4068B; Thu, 7 Apr 2022 18:05:52 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id D2DBF40689 for ; Thu, 7 Apr 2022 18:05:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649347552; x=1680883552; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=8SoZL7ykELrKRviKPwSXluaB5APLwCmFZb517jO5+/U=; b=WrHf4NHEwyCXCNk4UOwYkKPmdw1K1kaeX68jggpHCGMabmNzjkVGbdr/ um1I8Ase4YTRsdU5iQFOcp6RXybTQiYQguiZ8ul3KTW2yyDXiVEsabJCZ u6Jhufq2YzfXrE0KV1AKhhFbu/D92mbK/syOfRmkGeNSKUSj9df4VhEPf YJlEKovF9K8RZLZIIn1crpkjXg8O2RIVC5PkMq7hXdhVjX2RlSQ4vmSW6 QZR80lgz9mHtOxivRXAt9vqI/IESCs3h11qeBiPbXVWdZR8wRp2UgoDAF QkbBUG+AXlGzWdu1wJPy93xVl3tj+94SpJfrRWfX+ZSKXvXqz06ghcLeh w==; X-IronPort-AV: E=McAfee;i="6400,9594,10310"; a="324528557" X-IronPort-AV: E=Sophos;i="5.90,242,1643702400"; d="scan'208";a="324528557" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2022 09:04:22 -0700 X-IronPort-AV: E=Sophos;i="5.90,242,1643702400"; d="scan'208";a="571119288" Received: from bricha3-mobl.ger.corp.intel.com ([10.213.221.29]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 07 Apr 2022 09:04:19 -0700 Date: Thu, 7 Apr 2022 17:04:16 +0100 From: Bruce Richardson To: Maxime Coquelin Cc: Ilya Maximets , "Van Haaren, Harry" , Morten =?iso-8859-1?Q?Br=F8rup?= , "Pai G, Sunil" , "Stokes, Ian" , "Hu, Jiayu" , "Ferriter, Cian" , "ovs-dev@openvswitch.org" , "dev@dpdk.org" , "Mcnamara, John" , "O'Driscoll, Tim" , "Finn, Emma" Subject: Re: OVS DPDK DMA-Dev library/Design Discussion Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D86F80@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D86F82@smartserver.smartshare.dk> <94d817cb-8151-6644-c577-ed8b42d24337@redhat.com> <55c5a37f-3ee8-394b-8cff-e7daecb59f73@ovn.org> <7162eebf-c62b-c16e-92e6-ea350ac0d647@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7162eebf-c62b-c16e-92e6-ea350ac0d647@redhat.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Apr 07, 2022 at 05:46:32PM +0200, Maxime Coquelin wrote: > > > On 4/7/22 17:01, Ilya Maximets wrote: > > On 4/7/22 16:42, Van Haaren, Harry wrote: > > > > -----Original Message----- > > > > From: Ilya Maximets > > > > Sent: Thursday, April 7, 2022 3:40 PM > > > > To: Maxime Coquelin ; Van Haaren, Harry > > > > ; Morten Brørup ; > > > > Richardson, Bruce > > > > Cc: i.maximets@ovn.org; Pai G, Sunil ; Stokes, Ian > > > > ; Hu, Jiayu ; Ferriter, Cian > > > > ; ovs-dev@openvswitch.org; dev@dpdk.org; Mcnamara, > > > > John ; O'Driscoll, Tim ; > > > > Finn, Emma > > > > Subject: Re: OVS DPDK DMA-Dev library/Design Discussion > > > > > > > > On 4/7/22 16:25, Maxime Coquelin wrote: > > > > > Hi Harry, > > > > > > > > > > On 4/7/22 16:04, Van Haaren, Harry wrote: > > > > > > Hi OVS & DPDK, Maintainers & Community, > > > > > > > > > > > > Top posting overview of discussion as replies to thread become slower: > > > > > > perhaps it is a good time to review and plan for next steps? > > > > > > > > > > > >  From my perspective, it those most vocal in the thread seem to be in favour > > > > of the clean > > > > > > rx/tx split ("defer work"), with the tradeoff that the application must be > > > > aware of handling > > > > > > the async DMA completions. If there are any concerns opposing upstreaming > > > > of this method, > > > > > > please indicate this promptly, and we can continue technical discussions here > > > > now. > > > > > > > > > > Wasn't there some discussions about handling the Virtio completions with > > > > > the DMA engine? With that, we wouldn't need the deferral of work. > > > > > > > > +1 > > > > > > Yes there was, the DMA/virtq completions thread here for reference; > > > https://mail.openvswitch.org/pipermail/ovs-dev/2022-March/392908.html > > > > > > I do not believe that there is a viable path to actually implementing it, and particularly > > > not in the more complex cases; e.g. virtio with guest-interrupt enabled. > > > > > > The thread above mentions additional threads and various other options; none of which > > > I believe to be a clean or workable solution. I'd like input from other folks more familiar > > > with the exact implementations of VHost/vrings, as well as those with DMA engine expertise. > > > > I tend to trust Maxime as a vhost maintainer in such questions. :) > > > > In my own opinion though, the implementation is possible and concerns doesn't > > sound deal-breaking as solutions for them might work well enough. So I think > > the viability should be tested out before solution is disregarded. Especially > > because the decision will form the API of the vhost library. > > I agree, we need a PoC adding interrupt support to dmadev API using > eventfd, and adding a thread in Vhost library that polls for DMA > interrupts and calls vhost_vring_call if needed. > Hi Maxime, couple of questions, perhaps you can clarify. Firstly, why would an eventfd be needed for the interrupts, can they not just use the regular interrupt handling like other devices in DPDK, e.g. read on /dev/node. In terms of the new thread - what is this thread going to handle? Is it going to take interrupts from all dma operations and handle the completion/cleanup of all jobs for all queues once the DMA engine is finished? Or is it just going to periodically be woken up to check for the edge case if there are any virtio queues sleeping with interrupts enabled where there are completed - but unsignalled to the VM - packets? Regards, /Bruce