From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7402DA0548; Sat, 3 Apr 2021 11:45:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 050E040696; Sat, 3 Apr 2021 11:45:12 +0200 (CEST) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) by mails.dpdk.org (Postfix) with ESMTP id CD9E64067B for ; Sat, 3 Apr 2021 11:45:10 +0200 (CEST) Received: by mail-io1-f45.google.com with SMTP id z3so4732842ioc.8 for ; Sat, 03 Apr 2021 02:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=98OXD7GI37FzeOBkkkv2/abtxxmP9TwHTq1Qgb5hNXQ=; b=HSRyPtapXfhb2RtALRQxpPrXklh56IY3HpxOx0RwqRLKIE7e4N949jcnvhB1Y6d95F Ku71U1lxEjDhuXxfzkV2VnhhquDqqviZoqGXoFcGONAuSk/5hA/uGO6iJcy3Xw1d/zFe FTDD97ndthfIjQuBUnb33umXFKQkhpu9/qhgdkuLTBtRLvhl3y64iiRjok3BG7j3JFYS hhVg3M89d0Qu0NVzw6zY8mUtVTKy3hRbi8a2etRsSfYfZrIHzpg6zGLr2X0sNUteRYLq D00se4oKRpxlV7CiLVnEPnV93fpYCjZ+nNNcXtCWH4+17AHQWvN6ZCQMM7i9MN0AbEl5 iiFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=98OXD7GI37FzeOBkkkv2/abtxxmP9TwHTq1Qgb5hNXQ=; b=KVImZ0fyDHVU3T6hUIjjJOjTxxuABa/hISkJlq0KGHYs3u5EQwikFvqdUwxAKcN5zl Sf6GKKrIMCFw12HOfby/C2FKyI6S4zLFZflnZ/mxIXHNvuDk2nCoVYEMnG5t3ookjS0D jQQcYIxarQeDHGIy2o47if4/YDnB+t0mBhxBJAK99dEi+B9CIlO4sDDs2Xp8HgSgCm39 d7dnCcocVEvDToJttOSJxR0q8gA/9TFzjs65Gel+mjIrFEPGoztbXRcHL0SWifyJX27v up8zO6veWGWq21Uf+rmpy0KYv81TvOB6tx4nuqjQFwxeQDszEJGu9yult0uZSfJwbtMY DpFg== X-Gm-Message-State: AOAM532iFDKK5f035MGCtWR9izPQZyzkv+r6m1jyn8E16GdeOl/fjAMd 3vKVMJeHFT70I11jIfg4I5RP45Fek2lYPJiaAv4= X-Google-Smtp-Source: ABdhPJw8FMZhoTWuprAKonvvZk0gK63Xuv/pDttZibymnrYTIh1ZmHxqPOQi5asVy562tCWPh0kw4QXqXt4BUD39fbE= X-Received: by 2002:a02:c899:: with SMTP id m25mr15990657jao.113.1617443109947; Sat, 03 Apr 2021 02:45:09 -0700 (PDT) MIME-Version: 1.0 References: <20210330082212.707-1-pbhagavatula@marvell.com> <20210331093003.263-1-pbhagavatula@marvell.com> In-Reply-To: <20210331093003.263-1-pbhagavatula@marvell.com> From: Jerin Jacob Date: Sat, 3 Apr 2021 15:14:53 +0530 Message-ID: To: Pavan Nikhilesh Cc: Jerin Jacob , "Jayatheerthan, Jay" , Erik Gabriel Carrillo , "Gujjar, Abhinandan S" , "McDaniel, Timothy" , Hemant Agrawal , "Van Haaren, Harry" , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Liang Ma , dpdk-dev Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v10 0/8] Introduce event vectorization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Mar 31, 2021 at 3:00 PM wrote: > > From: Pavan Nikhilesh > > In traditional event programming model, events are identified by a > flow-id and a uintptr_t. The flow-id uniquely identifies a given event > and determines the order of scheduling based on schedule type, the > uintptr_t holds a single object. > > Event devices also support burst mode with configurable dequeue depth, > i.e. each dequeue call would return multiple events and each event > might be at a different stage of the pipeline. > Having a burst of events belonging to different stages in a dequeue > burst is not only difficult to vectorize but also increases the scheduler > overhead and application overhead of pipelining events further. > Using event vectors we see a performance gain of ~742.3% as shown in [1]. > > By introducing event vectorization, each event will be capable of holding > multiple uintptr_t of the same flow thereby allowing applications > to vectorize their pipeline and reduce the complexity of pipelining > events across multiple stages. This also reduces the complexity of handling > enqueue and dequeue on an event device. > > Since event devices are transparent to the events they are scheduling > so the event producers such as eth_rx_adapter, crypto_adapter , etc.. > are responsible for vectorizing the buffers of the same flow into a single > event. > > The series also breaks ABI in the patch [8/8] which is targetted to the > v21.11 release. > > The dpdk-test-eventdev application has been updated with options to test > multiple vector sizes and timeouts. > > [1] > As for performance improvement, with a ARM Cortex-A72 equivalent processer, > software event device (--vdev=event_sw0), single worker core, single stage > and using one service core for Rx adapter, Tx adapter, Scheduling. > > Without this patchset applied: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 5.071 mpps > > With the patchset applied and Without event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 5.123 mpps > > With event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 --enable_vector --nb_eth_queues 1 > --vector_size 256 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 42.715 mpps > > Having dedicated service cores for each Rx queues and tweaking the vector, > dequeue burst size would further improve performance. > > API usage is shown below: > > Configuration: > > struct rte_event_eth_rx_adapter_event_vector_config vec_conf; > > vector_pool = rte_event_vector_pool_create("vector_pool", > nb_elem, 0, vector_size, socket_id); > > rte_event_eth_rx_adapter_create(id, event_id, &adptr_conf); > rte_event_eth_rx_adapter_queue_add(id, eth_id, -1, &queue_conf); > if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR) { > vec_conf.vector_sz = vector_size; > vec_conf.vector_timeout_ns = vector_tmo_nsec; > vec_conf.vector_mp = vector_pool; > rte_event_eth_rx_adapter_queue_event_vector_config(id, > eth_id, -1, &vec_conf); > } > > Fastpath: > > num = rte_event_dequeue_burst(event_id, port_id, &ev, 1, 0); > if (!num) > continue; > > if (ev.event_type & RTE_EVENT_TYPE_VECTOR) { > switch (ev.event_type) { > case RTE_EVENT_TYPE_ETHDEV_VECTOR: > case RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR: > struct rte_mbuf **mbufs; > > mbufs = ev.vector_ev->mbufs; > for (i = 0; i < ev.vector_ev->nb_elem; i++) > //Process mbufs. > break; > case ... > } > } > ... > Series applied to dpdk-next-net-eventdev/for-main. Thanks > v10 Changes: > - Update Rx adapter documentation with flow identifier bitfield format. (Jay) > > v9 Changes: > - Update Rx adapter documentation w.r.t SW event vectorizations. (Jay) > - Push partial vectors to event device on queue delete. (Jay) > > v8 Changes: > - Fix incorrect shift for vector timeout interval.(Jay) > - Code reallocation.(Jay) > > v7 Changes: > - More doxygen fixes.(Jay) > - Reduce code duplication in 4/8.(Jay) > > v6 Changes: > - Make rte_errno sign consistant.(Jay) > - Gramatical and doxygen fixes. (Jay) > > v5 Changes: > - Make `rte_event_vector_pool_create non-inline` to ease ABI stability.(Ray) > - Move `rte_event_eth_rx_adapter_queue_event_vector_config` and > `rte_event_eth_rx_adapter_vector_limits_get` implementation to the patch > where they are initially defined.(Ray) > - Multiple gramatical and style fixes.(Jerin) > - Add missing release notes.(Jerin) > > v4 Changes: > - Fix missing event vector structure in event structure.(Jay) > > v3 Changes: > - Fix unintended formatting changes. > > v2 Changes: > - Multiple gramatical and style fixes.(Jerin) > - Add parameter to define vector size in power of 2. (Jerin) > - Redo patch series w/o breaking ABI till the last patch.(David) > - Add deprication notice to announce ABI break in 21.11.(David) > - Add vector limits validation to app/test-eventdev. > > Pavan Nikhilesh (8): > eventdev: introduce event vector capability > eventdev: introduce event vector Rx capability > eventdev: introduce event vector Tx capability > eventdev: add Rx adapter event vector support > eventdev: add Tx adapter event vector support > app/eventdev: add event vector mode in pipeline test > doc: announce event Rx adapter config changes > eventdev: simplify Rx adapter event vector config > > app/test-eventdev/evt_common.h | 4 + > app/test-eventdev/evt_options.c | 52 +++ > app/test-eventdev/evt_options.h | 4 + > app/test-eventdev/test_pipeline_atq.c | 310 +++++++++++++++-- > app/test-eventdev/test_pipeline_common.c | 105 +++++- > app/test-eventdev/test_pipeline_common.h | 18 + > app/test-eventdev/test_pipeline_queue.c | 320 +++++++++++++++-- > .../prog_guide/event_ethernet_rx_adapter.rst | 57 +++ > .../prog_guide/event_ethernet_tx_adapter.rst | 12 + > doc/guides/prog_guide/eventdev.rst | 36 +- > doc/guides/rel_notes/deprecation.rst | 9 + > doc/guides/rel_notes/release_21_05.rst | 8 + > doc/guides/tools/testeventdev.rst | 45 ++- > lib/librte_eventdev/eventdev_pmd.h | 31 +- > .../rte_event_eth_rx_adapter.c | 327 +++++++++++++++++- > .../rte_event_eth_rx_adapter.h | 78 +++++ > .../rte_event_eth_tx_adapter.c | 66 +++- > lib/librte_eventdev/rte_eventdev.c | 53 ++- > lib/librte_eventdev/rte_eventdev.h | 114 +++++- > lib/librte_eventdev/version.map | 4 + > 20 files changed, 1562 insertions(+), 91 deletions(-) > > -- > 2.17.1 >