From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 659D12C54 for ; Wed, 29 Nov 2017 18:15:17 +0100 (CET) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Nov 2017 09:15:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,338,1508828400"; d="scan'208";a="154428612" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga004.jf.intel.com with ESMTP; 29 Nov 2017 09:15:14 -0800 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id vATHFDDP020870; Wed, 29 Nov 2017 17:15:13 GMT Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id vATHFDdh010539; Wed, 29 Nov 2017 17:15:13 GMT Received: (from lma25@localhost) by sivswdev01.ir.intel.com with LOCAL id vATHFChj010509; Wed, 29 Nov 2017 17:15:12 GMT Date: Wed, 29 Nov 2017 17:15:12 +0000 From: "Ma, Liang" To: Jerin Jacob Cc: "dev@dpdk.org" , "Van Haaren, Harry" , "Richardson, Bruce" , "Jain, Deepak K" , "Mccarthy, Peter" Message-ID: <20171129171512.GA30238@sivswdev01.ir.intel.com> References: <1511522632-139652-1-git-send-email-liang.j.ma@intel.com> <20171124205532.GA5197@jerin> <20171129121954.GA23464@sivswdev01.ir.intel.com> <20171129125605.GA24298@jerin> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171129125605.GA24298@jerin> User-Agent: Mutt/1.9.1 (2017-09-22) Subject: Re: [dpdk-dev] [RFC PATCH 0/7] RFC:EventDev OPDL PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2017 17:15:19 -0000 On 29 Nov 04:56, Jerin Jacob wrote: > -----Original Message----- > > Date: Wed, 29 Nov 2017 12:19:54 +0000 > > From: "Ma, Liang" > > To: Jerin Jacob > > CC: dev@dpdk.org, "Van Haaren, Harry" , > > "Richardson, Bruce" , "Jain, Deepak K" > > , "Mccarthy, Peter" > > Subject: Re: [RFC PATCH 0/7] RFC:EventDev OPDL PMD > > User-Agent: Mutt/1.5.20 (2009-06-14) > > > > Hi Jerin, > > Many thanks for your comments. Please check my comment below. > > > > On 25 Nov 02:25, Jerin Jacob wrote: > > > -----Original Message----- > > > > Date: Fri, 24 Nov 2017 11:23:45 +0000 > > > > From: liang.j.ma@intel.com > > > > To: jerin.jacob@caviumnetworks.com > > > > CC: dev@dpdk.org, harry.van.haaren@intel.com, bruce.richardson@intel.com, > > > > deepak.k.jain@intel.com, john.geary@intel.com > > > > Subject: [RFC PATCH 0/7] RFC:EventDev OPDL PMD > > > > X-Mailer: git-send-email 2.7.5 > > > > > > > > From: Liang Ma > > > > > > > > > Thanks Liang Ma for the RFC. > > > > > > > > > > > The OPDL (Ordered Packet Distribution Library) eventdev is a specific > > > > implementation of the eventdev API. It is particularly suited to packet > > > > processing workloads that have high throughput and low latency > > > > requirements. All packets follow the same path through the device. > > > > The order which packets follow is determinted by the order in which > > > > queues are set up. Packets are left on the ring until they are transmitted. > > > > As a result packets do not go out of order. > > > > > > > > Features: > > > > > > > > The OPDL eventdev implements a subset of features of the eventdev API; > > > > > > > > Queues > > > > * Atomic > > > > * Ordered (Parallel is supported as parallel is a subset of Ordered) > > > > * Single-Link > > > > > > > > Ports > > > > * Load balanced (for Atomic, Ordered, Parallel queues) > > > > * Single Link (for single-link queues) > > > > > > > > Single Port Queue > > > > > > > > It is possible to create a Single Port Queue > > > > RTE_EVENT_QUEUE_CFG_SINGLE_LINK. Packets dequeued from this queue do > > > > not need to be re-enqueued (as is the case with an ordered queue). The > > > > purpose of this queue is to allow for asynchronous handling of packets in > > > > the middle of a pipeline. Ordered queues in the middle of a pipeline > > > > cannot delete packets. > > > > > > > > > > > > Queue Dependencies > > > > > > > > As stated the order in which packets travel through queues is static in > > > > nature. They go through the queues in the order the queues are setup at > > > > initialisation rte_event_queue_setup(). For example if an application > > > > sets up 3 queues, Q0, Q1, Q2 and has 3 assoicated ports P0, P1, P2 and > > > > P3 then packets must be > > > > > > > > * Enqueued onto Q0 (typically through P0), then > > > > > > > > * Dequeued from Q0 (typically through P1), then > > > > > > > > * Enqueued onto Q1 (also through P1), then > > > > > > > > * Dequeued from Q2 (typically through P2), then > > > > > > > > * Enqueued onto Q3 (also through P2), then > > > > > > > > * Dequeued from Q3 (typically through P3) and then transmitted on the > > > > relevant eth port > > > > > > > > > > > > Limitations > > > > > > > > The opdl implementation has a number of limitations. These limitations are > > > > due to the static nature of the underlying queues. It is because of this > > > > that the implementation can achieve such high throughput and low latency > > > > > > > > The following list is a comprehensive outline of the what is supported and > > > > the limitations / restrictions imposed by the opdl pmd > > > > > > > > - The order in which packets moved between queues is static and fixed > > > > (dynamic scheduling is not supported). > > > > > > > > - NEW, RELEASE op type are not explicitly supported. RX (first enqueue) > > > > implicitly adds NEW event types, and TX (last dequeue) implicitly does > > > > RELEASE event types. > > > > > > > > - All packets follow the same path through device queues. > > > > > > > > - Flows within queues are NOT supported. > > > > > > > > - Event priority is NOT supported. > > > > > > > > - Once the device is stopped all inflight events are lost. Applications should > > > > clear all inflight events before stopping it. > > > > > > > > - Each port can only be associated with one queue. > > > > > > > > - Each queue can have multiple ports associated with it. > > > > > > > > - Each worker core has to dequeue the maximum burst size for that port. > > > > > > > > - For performance, the rte_event flow_id should not be updated once > > > > packet is enqueued on RX. > > > > > > Some top-level comments, > > > > > > # How does application knows this PMD has above limitations? > > > > > > I think, We need to add more capability RTE_EVENT_DEV_CAP_* > > > to depict these constraints. On the same note, I believe this > > > PMD is "radically" different than other SW/HW PMD then anyway > > > we cannot write the portable application using this PMD. So there > > > is no point in abstracting it as eventdev PMD. Could you please > > > work on the new capabilities are required to enable this PMD. > > > If it needs more capability flags to express this PMD capability, > > > we might have a different library for this as it defects the > > > purpose of portable eventdev applications. > > > > > Agree with improve capability information with add more details with > > RTE_EVENT_DEV_CAP_*. While the OPDL is designed around a different > > Please submit patches required for new caps required for this PMD to > depict the constraints. That is the only way application can know > the constraints for the given PMD. > I will work on capability issue and submit V2 patches when that's ready. > > load-balancing architecture, that of load-balancing across pipeline > > stages where a consumer is only working on a single stage, this does not > > necessarily mean that it is completely incompatible with other eventdev > > implementations. Although, it is true that an application written to use > > one of the existing eventdevs probably won't work nicely with the OPDL > > eventdev, the converse situation should work ok. That is, an application > > written as a pipeline using the OPDL eventdev for load balancing should > > work without changes with the generic SW implementation, and there should > > be no reason why it should not also work with other HW implementations > > in DPDK too. > > OPDL PMD implement a subset functionality of eventdev API. I demonstrate > > OPDL on this year PRC DPDK summit, got some early feedback from potential > > users. Most of them would like to use that under existing API(aka eventdev) > > rather than another new API/lib. That let potential user easier to swap to > > exist SW/HW eventdev PMD. > > Perfect. Lets have one application then so it will it make easy to swap > SW/HW eventdev PMD. > > > > > > # We should not add yet another "PMD" specific example application > > > in example area like "examples/eventdev_pipeline_opdl_pmd". We are > > > working on making examples/eventdev/pipeline_sw_pmd to make work > > > on both HW and SW. > > > > > We would agree here that we don't need a proliferation of example applications. > > However this is a different architecture (not a dynamic packet scheduler rather > > a static pipeline work distributer), and as such perhaps we should have a > > sample app that demonstrates each contrasting architecture. > > I agree. We need sample application. Why not change the exiting > examples/eventdev/pipeline_sw_pmd to make it work as we are addressing the > pipeling here. Let write the application based on THE USE CASE not > specific to PMD. PMD specific application won't scale. > I perfer to pending upstream OPDL example code in this patches set. it's better to upstream/merge example code in another track. > > > > > # We should not add new PMD specific test cases in > > > test/test/test_eventdev_opdl.c area.I think existing PMD specific > > > test case can be moved to respective driver area, and it can do > > > the self-test by passing some command line arguments to vdev. > > > > > We simply followed the existing test structure here. Would it be confusing to > > have another variant of example test code, is this done anywhere else? > > Also would there be a chance that DTS would miss running tests or not like > > having to run them using a different method. However we would defer to the consensus here. > > Could you elaborate on your concerns with having another test file in the test area ? > > PMD specific test cases wont scale. It defect the purpose of the common > framework. Cryptodev fell into that trap earlier then they fixed it. > For DTS case, I think, still it can verified through vdev command line > arguments to the new PMD. What do you think? > Agree, I would like to intergrate the test code with PMD, but any API is avaiable for self test purpose ? I didn't find existing api support self test. any hints ? > > > > > # Do you have relative performance number with exiting SW PMD? > > > Meaning how much it improves for any specific use case WRT exiting > > > SW PMD. That should a metric to define the need for new PMD. > > > > > Yes, we definitely has the number. Given the limitation(Ref cover letter), OPDL > > can achieve 3X-5X times schedule rate(on Xeon 2699 v4 platform) compare with the > > standard SW PMD and no need of schedule core. This is the core value of OPDL PMD. > > For certain user case, "static pipeline" "strong order ", OPDL is very useful > > and efficient and generic to processor arch. > > Sounds good. > > > > > > # There could be another SW driver from another vendor like ARM. > > > So, I think, it is important to define the need for another SW > > > PMD and how much limitation/new capabilities it needs to define to > > > fit into the eventdev framework, > > > > > Put a summary here, OPDL is designed for certain user case, performance is increase > > dramatically. Also OPDL can fallback to standard SW PMD seamless. > > That definitely fit into the eventdev API > >