From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9D994A04E6; Sat, 31 Oct 2020 13:50:10 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3B58DCA64; Sat, 31 Oct 2020 13:50:08 +0100 (CET) Received: from mail-il1-f195.google.com (mail-il1-f195.google.com [209.85.166.195]) by dpdk.org (Postfix) with ESMTP id 27392CA57 for ; Sat, 31 Oct 2020 13:50:07 +0100 (CET) Received: by mail-il1-f195.google.com with SMTP id q1so8912875ilt.6 for ; Sat, 31 Oct 2020 05:50:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=NoNTmR1Rga1jbfGp2zpVPfqjNRL4aZ8Ru80a+KGH6xY=; b=SpqcHGRBcKbksx9UqTLDomsEsn1FTgiZEE9ESy+xfW/O+xN+Wagzk9TAq/1Y2JQd5Q yFlaB/pzE7q894W+haFKzes4SyCn4Z7ELX3PVMtnpl6HhUI/XD50cWFYwQOnuzZ8IDqK 3EXfKGcv/6/yRUmO98nWL53KdIykhWoRntoPIhrKsL4gi9ZZ2pWEa+jyDyfjw6FH96Bf +7YlPrgfBgsfBSriZG0BlzmUNGRrsxwmN3199XAPFmFbsIoAgpFa7MZzDhGCSlJ/wssZ C+cmRXEZjmhWSOus8+splwpr6ZX8Zb4xqLR2UIFem/dCyoLW+3jDmFOxQZ8E3+tJJj8T yg4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=NoNTmR1Rga1jbfGp2zpVPfqjNRL4aZ8Ru80a+KGH6xY=; b=fLYzfUQoOE29AOyVh6anKsRJXcJkbYEziEHV+hkFDGyXDGO/TYOHJCG6Vgcw6FuJZv h2bfw9C7lWGRclXGnsR8dTkQT2+IUtSAVQAFr5NRxJdA9ZAjywCpjTRfCxXUsANkzfys 9cOZpNlCzGsXQKfP7e3ZGWpfl7m5WgCYfY0lTX9k74bXzKkolImhVk19Sr2tdn9Tw/90 4ANeVkQsqPv6o2hm4gkw6c2dpJwER+nl81fxvcBjQCUCPiMnwbzKCnEOvoRSofITuxFa 0y9hHTl0ENMJqDFpgUNmNrTBsRmLGfMZrhElDzriewobJf8U3YcHfRVstjrsNVwQ0qcy 7uEA== X-Gm-Message-State: AOAM532LS6DrYLPoAtPKWTvpawFjs4ggvMlsPSgBNu2j49YQNSrZvdKQ oGR7FAkHsQqdz6jb23klMbgxItQ2Gqscffo5/YBVwx5fLadOxQ== X-Google-Smtp-Source: ABdhPJzWn3mgrk1fIUFvnnh+PKxFypL+MUX17NgVURgQYCpHx3kYTm7lXJF8cwELMqYhVlWU7lWwnzN8G6jUI7bb20A= X-Received: by 2002:a05:6e02:5d1:: with SMTP id l17mr5225125ils.130.1604148605206; Sat, 31 Oct 2020 05:50:05 -0700 (PDT) MIME-Version: 1.0 References: <20200612212434.6852-2-timothy.mcdaniel@intel.com> <1604110397-3012-1-git-send-email-timothy.mcdaniel@intel.com> In-Reply-To: <1604110397-3012-1-git-send-email-timothy.mcdaniel@intel.com> From: Jerin Jacob Date: Sat, 31 Oct 2020 18:19:49 +0530 Message-ID: To: Timothy McDaniel Cc: dpdk-dev , Erik Gabriel Carrillo , Gage Eads , "Van Haaren, Harry" , Jerin Jacob , Thomas Monjalon Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v13 00/23] Add DLB PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Sat, Oct 31, 2020 at 7:41 AM Timothy McDaniel wrote: > > The following patch series adds support for a new eventdev PMD. The DLB > PMD adds support for the Intel Dynamic Load Balancer (DLB) hardware. > The DLB is a PCIe device that provides load-balanced, prioritized > scheduling of core-to-core communication. The device consists of > queues and arbiters that connect producer and consumer cores, and > implements load-balanced queueing features including: > - Lock-free multi-producer/multi-consumer operation. > - Multiple priority levels for varying traffic types. > - 'Direct' traffic (i.e. multi-producer/single-consumer) > - Simple unordered load-balanced distribution. > - Atomic lock-free load balancing across multiple consumers. > - Queue element reordering feature allowing ordered load-balanced > distribution. > > The DLB hardware supports both load balanced and directed ports and > queues. Unlike other eventdev devices already in the repo, not all > DLB ports and queues are equally capable. In particular, directed > ports are limited to a single link, and must be connected to a directed > queue. > Additionally, even though LDB ports may link multiple queues, the > number of queues that may be linked is limited by hardware. Another > difference is that DLB does not have a straightforward way of carrying > the flow_id in the queue elements (QE) that the hardware operates on. > > While reviewing the code, please be aware that this PMD has full > control over the DLB hardware. Intel will be extending the DLB PMD > in the future (not as part of this first series) with a mode that we > refer to as the bifurcated PMD. The bifurcated PMD communicates with a > kernel driver to configure the device, ports, and queues, and memory > maps device MMIO so datapath operations occur purely in user-space. > > The framework to support both the PF PMD and bifurcated PMD exists in > this patchset, and is why the iface.[ch] layer is present. > > Major changes in V13 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - removed now unused functions dlb_umwait and dlb_umonitor build error with clang at "event/dlb: add enqueue and its burst variants" patch. Please make sure each patch builds to avoid delay in merging the patch. Also, address the David comment on the doc for the next version. FAILED: drivers/libtmp_rte_event_dlb.a.p/event_dlb_dlb.c.o ccache clang -Idrivers/libtmp_rte_event_dlb.a.p -Idrivers -I../drivers -Idrivers/event/dlb -I../drivers/event/dlb -Ilib/librte_eventdev -I../lib/librte_eventdev -I. -I.. -Iconfig -I../config -Ilib/librte_eal/include -I../lib/librte_eal/incl ude -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include -Ilib/librte_eal/common -I../lib/librte_eal/common -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs -I ../lib/librte_kvargs -Ilib/librte_metrics -I../lib/librte_metrics -Ilib/librte_telemetry -I../lib/librte_telemetry -Ilib/librte_ring -I../lib/librte_ring -Ilib/librte_ethdev -I../lib/librte_ethdev -Ilib/librte_net -I../lib/librte_net -Ilib/ librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool -I../lib/librte_mempool -Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_rcu -I../lib/librte_rcu -Ilib/librte_timer -I../lib/librte_timer -Ili b/librte_cryptodev -I../lib/librte_cryptodev -Ilib/librte_pci -I../lib/librte_pci -Idrivers/bus/pci -I../drivers/bus/pci -I../drivers/bus/pci/linux -Xclang -fcolor-diagnostics -pipe -D_FILE_OFFSET_BITS=3D64 -Wall -Winvalid-pch -Werror -O2 -g -include rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwr ite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -march=3Dnative -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -MD -MQ drivers/libtmp_rte_event_dlb.a.p/event_dlb_dlb.c.o -MF drivers/libtmp_rte_even t_dlb.a.p/event_dlb_dlb.c.o.d -o drivers/libtmp_rte_event_dlb.a.p/event_dlb_dlb.c.o -c ../drivers/event/dlb/dlb.c ../drivers/event/dlb/dlb.c:2777:1: error: unused function 'dlb_event_enqueue_delayed' [-Werror,-Wunused-function] dlb_event_enqueue_delayed(void *event_port, ^ ../drivers/event/dlb/dlb.c:2762:1: error: unused function 'dlb_event_enqueue_burst_delayed' [-Werror,-Wunused-function] dlb_event_enqueue_burst_delayed(void *event_port, ^ ../drivers/event/dlb/dlb.c:2792:1: error: unused function 'dlb_event_enqueue_new_burst_delayed' [-Werror,-Wunused-function] dlb_event_enqueue_new_burst_delayed(void *event_port, ^ ../drivers/event/dlb/dlb.c:2808:1: error: unused function 'dlb_event_enqueue_forward_burst_delayed' [-Werror,-Wunused-function] dlb_event_enqueue_forward_burst_delayed(void *event_port, ^ ../drivers/event/dlb/dlb.c:2605:1: error: unused function 'dlb_construct_token_pop_qe' [-Werror,-Wunused-function] dlb_construct_token_pop_qe(struct dlb_port *qm_port, int idx) ^ ../drivers/event/dlb/dlb.c:2653:1: error: unused function 'dlb_consume_qe_immediate' [-Werror,-Wunused-function] dlb_consume_qe_immediate(struct dlb_port *qm_port, int num) ^ 6 errors generated. > > Major changes in V12 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Fix CENTOS build error: use __m128i instead of __v2di with > _mm_stream_si128 > > Major changes in V11 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - removed unused function, fixing build error > - fixed typo in port_setup commit message > - this patch series is based on dpdk-next-eventdev > > Major changes in v10 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - convert to use rte_power_monitor patches > - replace __builtin_ia32_movntdq() with _mm_stream_si128() > - remove unused functions in dlb_selftest.c > > Major changes in v9 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - fixed a build error due to __rte_cache_aligned being placed after > the ";" character, instead of before it. > > Major changes in v8 after dpdk reviews > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - moved introduction of dlb in relnotes_20_11 to first patch in series > - fixed underlines in dlb.rst that were too short > - note that the code still uses its private byte-encoded versions of > umonitor/umwait, rather than the new functions in the power > patch that are built on top of those intrinsics. This is intentional. > > Major changes in v7 after dpdk reviews > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - updated MAINTAINERS file to alphabetically insert DLB > - don't create RTE_ symbols in PMD > - converted to use version.map scheme > - converted to use .._master_lcore instead of .._main_lcore > - this patch set is based on dpdk-next-eventdev > > Major changes in v6 after dpdk reviews: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - fixed meson conditional build. Moved test into driver=E2=80=99s meson.b= uild > file instead of event/meson.build > - documentation is populated as associated code is introduced > - add log_register in add dynamic logging patch > - rename RTE_xxx symbol(s) as DLB2_xxx > - replaced function ptr enqueue_four with direct call to movdir64b > - remove unused port_pages > - broke up probe patch into 3 smaller patches for easier review > - changed param order of movdir64b/movntdq to match intrinsics > - added self to MAINTAINERS files > - squashed announcement of availability into last patch in series > - correct spelling errors and delete repeated words > - DPDK_21.0 -> DPDK 21 in map file > - add experimental banner to public structs and APIs > - implemented other suggestions from code reviews of DLB2 PMD. The > software is very similar in form so some DLB2 reviews comments > were applicable to DLB as well > > Major changes in v5 after dpdk reviews and additional internal reviews > by colleagues at Intel: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - implement changes requested in code reviews by Gage Eads and Mike Chen > - fix a memzone leak > - convert to use eal rte-cpuflags patch from Liang Ma > > Major changes in v4 after dpdk reviews and additional internal reviews > by colleagues at Intel: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Remove make infrastructure > - shared code (pf/base) is now added incrementally > - flexible interface (iface.[ch]) is now added incrementally > - removed calls to rte_panic > - do not call pthread_create directly > - remove unused internal API, os_time > - convert rte_atomic to __atomic builtins > - broke out eventdev ABI changes, test/api changes, and new internal PCI > named probe API > - relocated enqueue logic to enqueue patch > > Major Changes in V3: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Fixed a memory corruption issue due to not allocating enough CQ > memory for depths < 8. Hardware requires minimum allocation to be > at least 8 entries. > - Address review comments from Gage and Mattias. > - Remove versioning > - minor formatting changes > > Major changes in V2: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - Correct ABI break that was present in V1. > - Address some of the review comments received from Mattias. > I will address the remaining items identified by Mattias in the next > patch delivery. > - General code cleanup based on internal code reviews > > Depends-on: patch-82202 ("eventdev: increase MAX QUEUES PER DEV to 255") > > Timothy McDaniel (23): > event/dlb: add documentation and meson infrastructure > event/dlb: add dynamic logging > event/dlb: add private data structures and constants > event/dlb: add definitions shared with LKM or shared code > event/dlb: add inline functions > event/dlb: add eventdev probe > event/dlb: add flexible interface > event/dlb: add probe-time hardware init > event/dlb: add xstats > event/dlb: add infos get and configure > event/dlb: add queue and port default conf > event/dlb: add queue setup > event/dlb: add port setup > event/dlb: add port link > event/dlb: add port unlink and port unlinks in progress > event/dlb: add eventdev start > event/dlb: add enqueue and its burst variants > event/dlb: add dequeue and its burst variants > event/dlb: add eventdev stop and close > event/dlb: add PMD's token pop public interface > event/dlb: add PMD self-tests > event/dlb: add queue and port release > event/dlb: add timeout ticks entry point > > MAINTAINERS | 6 +- > app/test/test_eventdev.c | 7 + > config/rte_config.h | 6 + > doc/api/doxy-api-index.md | 1 + > doc/guides/eventdevs/dlb.rst | 341 ++ > doc/guides/eventdevs/index.rst | 1 + > doc/guides/rel_notes/release_20_11.rst | 5 + > drivers/event/dlb/dlb.c | 4080 +++++++++++++++ > drivers/event/dlb/dlb_iface.c | 79 + > drivers/event/dlb/dlb_iface.h | 82 + > drivers/event/dlb/dlb_inline_fns.h | 40 + > drivers/event/dlb/dlb_log.h | 25 + > drivers/event/dlb/dlb_priv.h | 513 ++ > drivers/event/dlb/dlb_selftest.c | 1539 ++++++ > drivers/event/dlb/dlb_user.h | 814 +++ > drivers/event/dlb/dlb_xstats.c | 1222 +++++ > drivers/event/dlb/meson.build | 21 + > drivers/event/dlb/pf/base/dlb_hw_types.h | 334 ++ > drivers/event/dlb/pf/base/dlb_osdep.h | 310 ++ > drivers/event/dlb/pf/base/dlb_osdep_bitmap.h | 441 ++ > drivers/event/dlb/pf/base/dlb_osdep_list.h | 131 + > drivers/event/dlb/pf/base/dlb_osdep_types.h | 31 + > drivers/event/dlb/pf/base/dlb_regs.h | 2368 +++++++++ > drivers/event/dlb/pf/base/dlb_resource.c | 6904 ++++++++++++++++++++= ++++++ > drivers/event/dlb/pf/base/dlb_resource.h | 876 ++++ > drivers/event/dlb/pf/dlb_main.c | 586 +++ > drivers/event/dlb/pf/dlb_main.h | 47 + > drivers/event/dlb/pf/dlb_pf.c | 750 +++ > drivers/event/dlb/rte_pmd_dlb.c | 38 + > drivers/event/dlb/rte_pmd_dlb.h | 77 + > drivers/event/dlb/version.map | 9 + > drivers/event/meson.build | 2 +- > 32 files changed, 21684 insertions(+), 2 deletions(-) > create mode 100644 doc/guides/eventdevs/dlb.rst > create mode 100644 drivers/event/dlb/dlb.c > create mode 100644 drivers/event/dlb/dlb_iface.c > create mode 100644 drivers/event/dlb/dlb_iface.h > create mode 100644 drivers/event/dlb/dlb_inline_fns.h > create mode 100644 drivers/event/dlb/dlb_log.h > create mode 100644 drivers/event/dlb/dlb_priv.h > create mode 100644 drivers/event/dlb/dlb_selftest.c > create mode 100644 drivers/event/dlb/dlb_user.h > create mode 100644 drivers/event/dlb/dlb_xstats.c > create mode 100644 drivers/event/dlb/meson.build > create mode 100644 drivers/event/dlb/pf/base/dlb_hw_types.h > create mode 100644 drivers/event/dlb/pf/base/dlb_osdep.h > create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_bitmap.h > create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_list.h > create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_types.h > create mode 100644 drivers/event/dlb/pf/base/dlb_regs.h > create mode 100644 drivers/event/dlb/pf/base/dlb_resource.c > create mode 100644 drivers/event/dlb/pf/base/dlb_resource.h > create mode 100644 drivers/event/dlb/pf/dlb_main.c > create mode 100644 drivers/event/dlb/pf/dlb_main.h > create mode 100644 drivers/event/dlb/pf/dlb_pf.c > create mode 100644 drivers/event/dlb/rte_pmd_dlb.c > create mode 100644 drivers/event/dlb/rte_pmd_dlb.h > create mode 100644 drivers/event/dlb/version.map > > -- > 2.6.4 >