From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 92859A0A0A; Tue, 23 Mar 2021 19:45:04 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 702C140683; Tue, 23 Mar 2021 19:45:04 +0100 (CET) Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by mails.dpdk.org (Postfix) with ESMTP id 142EF4014D for ; Tue, 23 Mar 2021 19:45:03 +0100 (CET) Received: by mail-io1-f53.google.com with SMTP id k25so6677987iob.6 for ; Tue, 23 Mar 2021 11:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ddCvRgUKmb2N90GBnszYgTqK59c+AYXhpodv4XIBM1Q=; b=aC/Sm++jcZ77IWdvQaf5MyCu4aK/YufOXDTnWlnAduXG9IDmv1vjf4tFCQik9c0Xbg Md139+Z48Ct1Rl1BbpBBN/yBATQHxkpoNLGtDmGR5QoW4r9ktPYOZ0xQXGtSkRXiUOEM T17qqomUmQ0ZLONRqWLRBGJzRp1PKyslHl7lKzBgJnU4m+T1SJdkbVA5IAWJsj8P+9bR NUR4AiW90oX0///Hj8OUEclB3ewOJ4QocaO4hTUK0KdyNAgU+NG4WwDHU0dI+UOHDINN YfCEAUvXck68+n39h6Q/D5sbptXi9DMb7oi8HrmlFA5fHuA3grP4FThjyv8DbSU5xMuk 0qqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ddCvRgUKmb2N90GBnszYgTqK59c+AYXhpodv4XIBM1Q=; b=kyG7sWVdwKLB2Iq/Du+pneRuGYg3WfAG2B4GW3JjTnvZ0ffxETyKP7O0/QskyPrb1u sedPaPEO+Qz6vO9CkdsCw1zsjsckwJcyeV39lFrSydQLYj6FbH3sb/aDqQs3qDXmsan5 FO31K49seTLMtHJBKtLg1z094hXWWARlJXvG5kVkghR+aq9g3SfBG3EtgNgC4JoFkh/W vHrQN90lvGrZ5m7zyT4txt1GLzzR5G7SlXnFqI8JNyzRh/UwsTYaWeNtqnwjE6pLznNO WuUQiHnkVx41deWo4+ehLN8w8kgHFt2jcU1uCuhK+5VR4D+Z3xn/aXmEla0zNSAVIXyN dqgA== X-Gm-Message-State: AOAM531fB3R1eSzKn+mivJENT5nNkNQw485F9xmpLAw/j0tES5hYMtA7 W8SX2eLAYPh/8mRLzGLnX3gEyo5dYWyKDa64nfs= X-Google-Smtp-Source: ABdhPJxz3jMfXDN2KBGBN1MNNKYpvnKRFG9pDSogfR22sSmFD8r2jW8j9IrhXlfsUvsy2FzfulekCMLwrvz4ioSWf7A= X-Received: by 2002:a05:6602:2d95:: with SMTP id k21mr5424843iow.123.1616525102462; Tue, 23 Mar 2021 11:45:02 -0700 (PDT) MIME-Version: 1.0 References: <20210316200156.252-1-pbhagavatula@marvell.com> <20210319205718.1436-1-pbhagavatula@marvell.com> In-Reply-To: <20210319205718.1436-1-pbhagavatula@marvell.com> From: Jerin Jacob Date: Wed, 24 Mar 2021 00:14:46 +0530 Message-ID: To: Pavan Nikhilesh Cc: Jerin Jacob , "Jayatheerthan, Jay" , Erik Gabriel Carrillo , "Gujjar, Abhinandan S" , "McDaniel, Timothy" , Hemant Agrawal , "Van Haaren, Harry" , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Liang Ma , dpdk-dev Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v4 0/8] Introduce event vectorization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Sat, Mar 20, 2021 at 2:27 AM wrote: > > From: Pavan Nikhilesh > > In traditional event programming model, events are identified by a > flow-id and a uintptr_t. The flow-id uniquely identifies a given event > and determines the order of scheduling based on schedule type, the > uintptr_t holds a single object. > > Event devices also support burst mode with configurable dequeue depth, > i.e. each dequeue call would return multiple events and each event > might be at a different stage of the pipeline. > Having a burst of events belonging to different stages in a dequeue > burst is not only difficult to vectorize but also increases the scheduler > overhead and application overhead of pipelining events further. > Using event vectors we see a performance gain of ~628% as shown in [1]. > > By introducing event vectorization, each event will be capable of holding > multiple uintptr_t of the same flow thereby allowing applications > to vectorize their pipeline and reduce the complexity of pipelining > events across multiple stages. This also reduces the complexity of handling > enqueue and dequeue on an event device. > > Since event devices are transparent to the events they are scheduling > so the event producers such as eth_rx_adapter, crypto_adapter , etc.. > are responsible for vectorizing the buffers of the same flow into a single > event. > > The series also breaks ABI in the patch [8/8] which is targetted to the > v21.11 release. > > The dpdk-test-eventdev application has been updated with options to test > multiple vector sizes and timeouts. > > [1] > As for performance improvement, with a ARM Cortex-A72 equivalent processer, > software event device (--vdev=event_sw0), single worker core, single stage > and using one service core for Rx adapter, Tx adapter, Scheduling. > > Without event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 4.728 mpps avg 4.728 mpps > > With event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 --enable_vector --nb_eth_queues 1 > --vector_size 256 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 34.383 mpps avg 34.383 mpps > > Having dedicated service cores for each Rx queues and tweaking the vector, > dequeue burst size would further improve performance. > > API usage is shown below: > > Configuration: > > struct rte_event_eth_rx_adapter_event_vector_config vec_conf; > > vector_pool = rte_event_vector_pool_create("vector_pool", > nb_elem, 0, vector_size, socket_id); > > rte_event_eth_rx_adapter_create(id, event_id, &adptr_conf); > rte_event_eth_rx_adapter_queue_add(id, eth_id, -1, &queue_conf); > if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR) { > vec_conf.vector_sz = vector_size; > vec_conf.vector_timeout_ns = vector_tmo_nsec; > vec_conf.vector_mp = vector_pool; > rte_event_eth_rx_adapter_queue_event_vector_config(id, > eth_id, -1, &vec_conf); > } > > Fastpath: > > num = rte_event_dequeue_burst(event_id, port_id, &ev, 1, 0); > if (!num) > continue; > > if (ev.event_type & RTE_EVENT_TYPE_VECTOR) { > switch (ev.event_type) { > case RTE_EVENT_TYPE_ETHDEV_VECTOR: > case RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR: > struct rte_mbuf **mbufs; > > mbufs = ev.vector_ev->mbufs; > for (i = 0; i < ev.vector_ev->nb_elem; i++) > //Process mbufs. > break; > case ... > } > } > ... > > v4 Changes: > - Fix missing event vector structure in event structure.(Jay) > > v3 Changes: > - Fix unintended formatting changes. > > v2 Changes: > - Multiple gramatical and style fixes.(Jerin) > - Add parameter to define vector size in power of 2. (Jerin) > - Redo patch series w/o breaking ABI till the last patch.(David) > - Add deprication notice to announce ABI break in 21.11.(David) > - Add vector limits validation to app/test-eventdev. > > Pavan Nikhilesh (8): > eventdev: introduce event vector capability > eventdev: introduce event vector Rx capability > eventdev: introduce event vector Tx capability > eventdev: add Rx adapter event vector support > eventdev: add Tx adapter event vector support > app/eventdev: add event vector mode in pipeline test > doc: announce event Rx adapter config changes > eventdev: simplify Rx adapter event vector config > > app/test-eventdev/evt_common.h | 4 + > app/test-eventdev/evt_options.c | 52 +++ > app/test-eventdev/evt_options.h | 4 + > app/test-eventdev/test_pipeline_atq.c | 310 +++++++++++++++-- > app/test-eventdev/test_pipeline_common.c | 105 +++++- > app/test-eventdev/test_pipeline_common.h | 18 + > app/test-eventdev/test_pipeline_queue.c | 320 ++++++++++++++++-- > .../prog_guide/event_ethernet_rx_adapter.rst | 38 +++ > .../prog_guide/event_ethernet_tx_adapter.rst | 12 + > doc/guides/prog_guide/eventdev.rst | 36 +- > doc/guides/rel_notes/deprecation.rst | 9 + > doc/guides/tools/testeventdev.rst | 28 ++ > lib/librte_eventdev/eventdev_pmd.h | 31 +- > .../rte_event_eth_rx_adapter.c | 305 ++++++++++++++++- > .../rte_event_eth_rx_adapter.h | 68 ++++ > .../rte_event_eth_tx_adapter.c | 66 +++- > lib/librte_eventdev/rte_eventdev.c | 11 +- > lib/librte_eventdev/rte_eventdev.h | 144 +++++++- > lib/librte_eventdev/version.map | 4 + > 19 files changed, 1479 insertions(+), 86 deletions(-) Please update release notes(doc/guides/rel_notes/release_21_05.rst) for this feature. If there are no more comments on this series from others. IMO, Good to merge the next series for RC1. > > -- > 2.17.1 >