From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7A94A0508; Tue, 29 Mar 2022 14:51:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B4ACF40691; Tue, 29 Mar 2022 14:51:09 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id A088740141 for ; Tue, 29 Mar 2022 14:51:08 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01D8436B.A63DEE9D" Subject: RE: OVS DPDK DMA-Dev library/Design Discussion Date: Tue, 29 Mar 2022 14:51:02 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D86F7C@smartserver.smartshare.dk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: OVS DPDK DMA-Dev library/Design Discussion Thread-Index: Adg/jDNGcC8G4wWtSxeVfUOuAS3Y6wACLuFQAM6falAAJoZBIA== References: From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Pai G, Sunil" , "Stokes, Ian" , "Hu, Jiayu" , "Ferriter, Cian" , "Van Haaren, Harry" , "Ilya Maximets" , , , Cc: "Mcnamara, John" , "O'Driscoll, Tim" , "Finn, Emma" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a multi-part message in MIME format. ------_=_NextPart_001_01D8436B.A63DEE9D Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Having thought more about it, I think that a completely different = architectural approach is required: =20 Many of the DPDK Ethernet PMDs implement a variety of RX and TX packet = burst functions, each optimized for different CPU vector instruction = sets. The availability of a DMA engine should be treated the same way. = So I suggest that PMDs copying packet contents, e.g. memif, pcap, = vmxnet3, should implement DMA optimized RX and TX packet burst = functions. =20 Similarly for the DPDK vhost library. =20 In such an architecture, it would be the application's job to allocate = DMA channels and assign them to the specific PMDs that should use them. = But the actual use of the DMA channels would move down below the = application and into the DPDK PMDs and libraries. =20 =20 Med venlig hilsen / Kind regards, -Morten Br=F8rup =20 From: Pai G, Sunil [mailto:sunil.pai.g@intel.com]=20 Sent: Monday, 28 March 2022 20.19 To: Stokes, Ian; Hu, Jiayu; Ferriter, Cian; Van Haaren, Harry; Ilya = Maximets; Maxime Coquelin (maxime.coquelin@redhat.com); = ovs-dev@openvswitch.org; dev@dpdk.org Cc: Mcnamara, John; O'Driscoll, Tim; Finn, Emma Subject: RE: OVS DPDK DMA-Dev library/Design Discussion =20 Hi All,=20 =20 Please see below PDF which will be presented in the call. https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%= 20vhost%20async%20datapath%20design%202022%20session%202.pdf = =20 =20 Thanks and Regards, Sunil =20 =20 -----Original Appointment----- From: Stokes, Ian =20 Sent: Thursday, March 24, 2022 9:07 PM To: Pai G, Sunil; Hu, Jiayu; Ferriter, Cian; Van Haaren, Harry; Ilya = Maximets; Maxime Coquelin (maxime.coquelin@redhat.com); = ovs-dev@openvswitch.org; dev@dpdk.org Cc: Mcnamara, John; O'Driscoll, Tim; Finn, Emma Subject: OVS DPDK DMA-Dev library/Design Discussion When: Tuesday, March 29, 2022 2:00 PM-3:00 PM (UTC+00:00) Dublin, = Edinburgh, Lisbon, London. Where: Google Meet =20 =20 Hi All, =20 This meeting is a follow up to the call earlier this week. =20 This week Sunil presented 3 different approaches to integrating DMA-Dev = with OVS along with the performance impacts. =20 https://github.com/Sunil-Pai-G/OVS-DPDK-presentation-share/blob/main/OVS%= 20vhost%20async%20datapath%20design%202022.pdf = =20 =20 The approaches were as follows: =20 =B7 Defer work. =B7 Tx completions from Rx context. =B7 Tx completions from Rx context + lockless ring. =20 The pros and cons of each approach were discussed but there was no clear = solution reached. =20 As such a follow up call was suggested to continue discussion and to = reach a clear decision on the approach to take. =20 Please see agenda as it stands below: =20 Agenda =B7 Opens =B7 Continue discussion of 3x approaches from last week (Defer = work, "V3", V4, links to patches in Sunil's slides above) =B7 Design Feedback (please review solutions of above & = slide-deck from last week before call to be informed) =B7 Dynamic Allocation of DMA engine per queue =B7 Code Availability (DPDK GitHub, OVS GitHub branches) =20 Please feel free to respond with any other items to be added to the = agenda. =20 Google Meet: https://meet.google.com/hme-pygf-bfb = =20 =20 Regards Ian =20 ------_=_NextPart_001_01D8436B.A63DEE9D Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Having thought more about it, I think that a completely different = architectural approach is required:

 

Many of the DPDK Ethernet PMDs implement a variety of RX and TX = packet burst functions, each optimized for different CPU vector = instruction sets. The availability of a DMA engine should be treated the = same way. So I suggest that PMDs copying packet contents, e.g. memif, = pcap, vmxnet3, should implement DMA optimized RX and TX packet burst = functions.

 

Similarly for the DPDK vhost library.

 

In such an architecture, it would be the application’s job to = allocate DMA channels and assign them to the specific PMDs that should = use them. But the actual use of the DMA channels would move down below = the application and into the DPDK PMDs and = libraries.

 

 

Med venlig hilsen / Kind regards,

-Morten Br=F8rup

 

From:= = Pai G, Sunil [mailto:sunil.pai.g@intel.com]
Sent: Monday, 28 = March 2022 20.19
To: Stokes, Ian; Hu, Jiayu; Ferriter, Cian; = Van Haaren, Harry; Ilya Maximets; Maxime Coquelin = (maxime.coquelin@redhat.com); ovs-dev@openvswitch.org; = dev@dpdk.org
Cc: Mcnamara, John; O'Driscoll, Tim; Finn, = Emma
Subject: RE: OVS DPDK DMA-Dev library/Design = Discussion

 

Hi All, =

 

Please see = below PDF which will be presented in the = call.

 

Thanks and = Regards,

Sunil

 

 

-----Origin= al Appointment-----
From: Stokes, Ian = <ian.stokes@intel.com>
Sent: Thursday, March 24, 2022 = 9:07 PM
To: Pai G, Sunil; Hu, Jiayu; Ferriter, Cian; Van = Haaren, Harry; Ilya Maximets; Maxime Coquelin = (maxime.coquelin@redhat.com); ovs-dev@openvswitch.org; = dev@dpdk.org
Cc: Mcnamara, John; O'Driscoll, Tim; Finn, = Emma
Subject: OVS DPDK DMA-Dev library/Design = Discussion
When: Tuesday, March 29, 2022 2:00 PM-3:00 PM = (UTC+00:00) Dublin, Edinburgh, Lisbon, London.
Where: Google = Meet

 

 

Hi = All,

 

This = meeting is a follow up to the call earlier this = week.

 

This week = Sunil presented 3 different approaches to integrating DMA-Dev with OVS = along with the performance impacts.

 

 

The = approaches were as follows:

 

=B7         = Defer = work.

=B7         = Tx = completions from Rx context.

=B7         = Tx = completions from Rx context + lockless = ring.

 

The pros = and cons of each approach were discussed but there was no clear solution = reached.

 

As such a = follow up call was suggested to continue discussion and to reach a clear = decision on the approach to take.

 

Please see = agenda as it stands below:

 

Agenda

=B7         = Opens<= /o:p>

=B7         = Continue = discussion of 3x approaches from last week (Defer work, = “V3”, V4, links to patches in Sunil’s slides = above)

=B7         = Design Feedback (please review solutions = of above & slide-deck from last week before call to be = informed)=

=B7         = Dynamic = Allocation of DMA engine per queue

=B7         = Code = Availability (DPDK GitHub, OVS GitHub = branches)

 

Please = feel free to respond with any other items to be added to the = agenda.

 

 

Regards

Ian<= o:p>

 

------_=_NextPart_001_01D8436B.A63DEE9D--