From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB125A0566; Mon, 8 Mar 2021 17:42:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 671A822A409; Mon, 8 Mar 2021 17:42:04 +0100 (CET) Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by mails.dpdk.org (Postfix) with ESMTP id A4E4D22A3B2 for ; Mon, 8 Mar 2021 17:42:03 +0100 (CET) Received: by mail-io1-f53.google.com with SMTP id a7so10565630iok.12 for ; Mon, 08 Mar 2021 08:42:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fATw1anfghUL2hU4UuaBjESA760KIriiu9kidqeiSuc=; b=St+f3II5BKRg0w58Wnax159J/ZYQ2/lJ4j4fDGyT9ayQR5orYOA12Jgr3fOeC57hIl sJiL0zZG1hbeQgpnap3qx7HeNrSis2VucJTWB0jb62AqvsF0yUk4bViIMtoH0TNUrXES w6x8ifDKWGetRrpfE0k56mADDdPKRZa04IhHWmmbPZMZC6Ln8WC6Iw5BB/iPsleiKHTH YuLhIRsUEoVCE+9JvyGRJX4BMxLNQo6iEcoSVHFYktId69GxozjpEgiepnb1p2x/FzxY eYV+ofsLiR90d/6tIqMIddxGd5uVmP0EPr6oWIoR6VM+CdyxnWzOXQIyiBZ4jZfYO1lW zpvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fATw1anfghUL2hU4UuaBjESA760KIriiu9kidqeiSuc=; b=DcER9BVwbpgYRwQ4uGl7iyXjxcPKroA1mjCSjGA3EAnrsvP5xDg/lHh7zOVj9Hf46w LcdIE0rSX5rcsiggOZt6o00CsJMRqDx+hfOqtW1DnEGEN7vkqZIeU/IQPRGlqsF3KA+7 xQjg8j9J/Spp+AinvJV4aj5J8qt2h+v0S7LpTsQW+42VFRZpim+DjWypTR5vWmpuEsqm 7GuXvEw9jkDaLj1tCyTjJaX2sZbOrdZGwNguM6nZ4SIWEfkHzHV0BRbHjlzuwmyp/NmC 9ZZNA7Sw+EYRXzYLsevMrkCNBuXRenxhOx7dzs11IbDjRCMX8/YesYUQco8ro9NxQ9+T j7xA== X-Gm-Message-State: AOAM531h6MsomuxOX1G2sfvo4opoEVLwop3XOgkIYKqoAvzluaOeVSNN xp7X0ha2e6N517gez120lfs66iD29A4E0D0OUgM= X-Google-Smtp-Source: ABdhPJzdfptfJ8liFQLL3MZi0nJpjKZDsn8zgTn2PI+0QRQK+KqMS2eQ51AtTt5eQmyG9+d1SYKQ7DGiiV98MVigJho= X-Received: by 2002:a05:6638:582:: with SMTP id a2mr23969502jar.133.1615221722738; Mon, 08 Mar 2021 08:42:02 -0800 (PST) MIME-Version: 1.0 References: <20210220220957.4583-1-pbhagavatula@marvell.com> In-Reply-To: <20210220220957.4583-1-pbhagavatula@marvell.com> From: Jerin Jacob Date: Mon, 8 Mar 2021 22:11:46 +0530 Message-ID: To: Pavan Nikhilesh Cc: Jerin Jacob , "Jayatheerthan, Jay" , Erik Gabriel Carrillo , "Gujjar, Abhinandan S" , "McDaniel, Timothy" , Hemant Agrawal , "Van Haaren, Harry" , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Liang Ma , dpdk-dev Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH 0/7] Introduce event vectorization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Sun, Feb 21, 2021 at 3:40 AM wrote: > > From: Pavan Nikhilesh > > In traditional event programming model, events are identified by a > flow-id and a uintptr_t. The flow-id uniquely identifies a given event > and determines the order of scheduling based on schedule type, the > uintptr_t holds a single object. > > Event devices also support burst mode with configurable dequeue depth, > i.e. each dequeue call would return multiple events and each event > might be at a different stage of the pipeline. > Having a burst of events belonging to different stages in a dequeue > burst is not only difficult to vectorize but also increases the scheduler > overhead and application overhead of pipelining events further. > Using event vectors we see a performance gain of ~150% as shown in [1]. Since it is from 4.72 to 34.383, it is 628%. We had an internal review of this series before it is posting to ML. So my comments will be limited on this series. Is anyone planning to review this patch series or any objection to introducing this feature? CC list already has all eventdev stakeholders. > > By introducing event vectorization, each event will be capable of holding > multiple uintptr_t of the same flow thereby allowing applications > to vectorize their pipeline and reduce the complexity of pipelining > events across multiple stages. This also reduces the complexity of handling > enqueue and dequeue on an event device. > > Since event devices are transparent to the events they are scheduling > so the event producers such as eth_rx_adapter, crypto_adapter , etc.. > are responsible for vectorizing the buffers of the same flow into a single > event. > > The series also breaks ABI in [2/7] patch which we fix in [7/7]. The patch > [7/7] can be changed in the next major release i.e. v21.11. > > The dpdk-test-eventdev application has been updated with options to test > multiple vector sizes and timeouts. > > [1] > As for performance improvement, with a ARM Cortex-A72 equivalent processer, > software event device (--vdev=event_sw0), single worker core, single stage > and using one service core for Rx adapter, Tx adapter, Scheduling. > > Without event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 4.728 mpps avg 4.728 mpps > > With event vectorization: > ./build/app/dpdk-test-eventdev -l 7-23 -s 0x700 --vdev="event_sw0" -- > --prod_type_ethdev --nb_pkts=0 --verbose 2 --test=pipeline_queue > --stlist=a --wlcores=20 --enable_vector --nb_eth_queues 1 > --vector_size 256 > Port[0] using Rx adapter[0] configured > Port[0] using Tx adapter[0] Configured > 34.383 mpps avg 34.383 mpps > > Having dedicated service cores for each Rx queues and tweaking the vector, > dequeue burst size would further improve performance. > > API usage is shown below: > > Configuration: > > struct rte_event_eth_rx_adapter_event_vector_config vec_conf; > > vector_pool = rte_event_vector_pool_create("vector_pool", > nb_elem, 0, vector_size, socket_id); > > rte_event_eth_rx_adapter_create(id, event_id, &adptr_conf); > rte_event_eth_rx_adapter_queue_add(id, eth_id, -1, &queue_conf); > if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR) { > vec_conf.vector_sz = vector_size; > vec_conf.vector_timeout_ns = vector_tmo_nsec; > vec_conf.vector_mp = vector_pool; > rte_event_eth_rx_adapter_queue_event_vector_config(id, > eth_id, -1, &vec_conf); > } > > Fastpath: > > num = rte_event_dequeue_burst(event_id, port_id, &ev, 1, 0); > if (!num) > continue; > > if (ev.event_type & RTE_EVENT_TYPE_VECTOR) { > switch (ev.event_type) { > case RTE_EVENT_TYPE_ETHDEV_VECTOR: > case RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR: > struct rte_mbuf **mbufs; > > mbufs = ev.vector_ev->mbufs; > for (i = 0; i < ev.vector_ev->nb_elem; i++) > //Process mbufs. > break; > case ... > } > } > ... > > Pavan Nikhilesh (7): > eventdev: introduce event vector capability > eventdev: introduce event vector Rx capability > eventdev: introduce event vector Tx capability > eventdev: add Rx adapter event vector support > eventdev: add Tx adapter event vector support > app/eventdev: add event vector mode in pipeline test > eventdev: fix ABI breakage due to event vector > > app/test-eventdev/evt_common.h | 4 + > app/test-eventdev/evt_options.c | 52 +++ > app/test-eventdev/evt_options.h | 4 + > app/test-eventdev/test_pipeline_atq.c | 310 +++++++++++++-- > app/test-eventdev/test_pipeline_common.c | 77 +++- > app/test-eventdev/test_pipeline_common.h | 18 + > app/test-eventdev/test_pipeline_queue.c | 320 +++++++++++++-- > .../prog_guide/event_ethernet_rx_adapter.rst | 38 ++ > .../prog_guide/event_ethernet_tx_adapter.rst | 12 + > doc/guides/prog_guide/eventdev.rst | 36 +- > doc/guides/tools/testeventdev.rst | 28 ++ > lib/librte_eventdev/eventdev_pmd.h | 60 ++- > .../rte_event_eth_rx_adapter.c | 367 +++++++++++++++++- > .../rte_event_eth_rx_adapter.h | 93 +++++ > .../rte_event_eth_tx_adapter.c | 66 +++- > lib/librte_eventdev/rte_eventdev.c | 11 +- > lib/librte_eventdev/rte_eventdev.h | 145 ++++++- > lib/librte_eventdev/version.map | 5 + > 18 files changed, 1560 insertions(+), 86 deletions(-) > > -- > 2.17.1 >