From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3DEA4A034E; Thu, 23 Apr 2020 16:24:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 70D281C130; Thu, 23 Apr 2020 16:24:10 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id E2B474CBD for ; Thu, 23 Apr 2020 16:24:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587651848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G3TZUmV8WQh5AI+b0CTbI0dvefwwffPGYX3m0on1Hyk=; b=jKFdOSTX90bnxRixRiabNbVGRX0j17CaN1piKY96ccJ/53n7uTfTzf6qmQBf0Yfa5ufIR/ pekjA1oqmR7kOuj9Hl2L69/AWAi5TLJDxST2HoK6rNH5I8f6fE1n7HJQejic4CDJJeBX76 ESkxlsnYcGbsKueHGPzztlCq2ncwbS0= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-445-cNuSrUawP96UcpMcjU61dw-1; Thu, 23 Apr 2020 10:24:03 -0400 X-MC-Unique: cNuSrUawP96UcpMcjU61dw-1 Received: by mail-vk1-f200.google.com with SMTP id s206so2795440vke.20 for ; Thu, 23 Apr 2020 07:24:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=c2MdYsZgTM3sun45fUyXhSp0yZIrWQBzgrIDrT7hhAo=; b=nH3VpqOfb+XLznoSeTWGQ90+eLkBP7/FlQoGzi9fRQd6N2pV451YKId28brbEAgGRy kglDnirh2K9+Gsqi2TlfYY6/huWgb7MmT4qh0dHXOUhWIGZbdX9Bxt8gO8piLcQUH3sH PQ1V09Ue32aQ2eRl7OwfxoIZn2fFKjO4Y06//5eBS8gtia2OKb+Z2s7C/DX7sN0ktj6h /4HPPB7lki5Z8Gx9/c+mEywjrthXm5XWCkzHpubp8DndChPc6QHHdQPbrMsnCPOGyQ2W 075A7KNNih8aOF5ovzAx0kEweqUGnYJ8QvmvA/J8OfGaR1JbvOdNfjkU++QTxmQEYkpo svNw== X-Gm-Message-State: AGi0PubXxFnZN9MSDxcCYxidQMzFNj2ALl9nbHN6kzEg55DNqMfLTHk8 mHJJKG8b8dAvD9XBuVJprapr/ibcSxlpYDe9PL4K+pcuWHvh+6k+BXeMrsrqFvc2lvLwD7DZDzV pelaSQdprVH7YJRCoh2c= X-Received: by 2002:a1f:3649:: with SMTP id d70mr3602998vka.12.1587651842657; Thu, 23 Apr 2020 07:24:02 -0700 (PDT) X-Google-Smtp-Source: APiQypJ7PzOhoUMLv+LYdi23QYKufLg/4/P6XAvIoOz7Cr9mqY5ZEO8Uf3YHc8WV0pbpbYI16BqFNbb10TNuhO9Bjc0= X-Received: by 2002:a1f:3649:: with SMTP id d70mr3602973vka.12.1587651842292; Thu, 23 Apr 2020 07:24:02 -0700 (PDT) MIME-Version: 1.0 References: <20200419100133.3232316-1-jerinj@marvell.com> <20200422190349.3768487-1-jerinj@marvell.com> In-Reply-To: <20200422190349.3768487-1-jerinj@marvell.com> From: David Marchand Date: Thu, 23 Apr 2020 16:23:50 +0200 Message-ID: To: Jerin Jacob Kollanukkaran Cc: dev , Thomas Monjalon , Bruce Richardson , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Sunil Kumar Kori X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v7 00/32] DPDK Trace support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Apr 22, 2020 at 9:04 PM wrote: > This patch set contains > ~~~~~~~~~~~~~~~~~~~~~~~~ > > # The native implementation of common trace format(CTF)[1] based tracer > # Public API to create the trace points. > # Add tracepoints to eal, ethdev, mempool, eventdev and cryptodev > library for tracing support > # A unit test case > # Performance test case to measure the trace overhead. (See eal/trace: > # add trace performance test cases, patch) > # Programmers guide for Trace support(See doc: add trace library guide, > # patch) > > # Tested OS: > ~~~~~~~~~~~ > - Linux > - FreeBSD > > # Tested open source CTF trace viewers > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > - Babeltrace > - Tracecompass > > # Trace overhead comparison with LTTng > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > trace overhead data on x86:[2] > # 236 cycles with LTTng(>100ns) > # 18 cycles(7ns) with Native DPDK CTF emitter.(See eal/trace: add trace > # performance test cases patch) > > trace overhead data on arm64: > # 312 cycles to 1100 cycles with LTTng based on the class of arm64 CP= U. > # 11 cycles to 13 cycles with Native DPDK CTF emitter based on the > class of arm64 CPU. > > 18 cycles(on x86) vs 11 cycles(on arm64) is due to rdtsc() overhead in > x86. It seems rdtsc takes around 15cycles in x86. > > More details: > ~~~~~~~~~~~~~ > > # The Native DPDK CTF trace support does not have any dependency on > third-party library. > The generated output file is compatible with LTTng as both are using > CTF trace format. > > The performance gain comes from: > 1) exploit dpdk worker thread usage model to avoid atomics and use per > core variables > 2) use hugepage, > 3) avoid a lot function pointers in fast-path etc > 4) avoid unaligned store for arm64 etc > > Features: > ~~~~~~~~~ > - No specific limit on the events. A string-based event like rte_log > for pattern matching > - Dynamic enable/disable support. > - Instructmention overhead is ~1 cycle. i.e cost of adding the code > wth out using trace feature. > - Timestamp support for all the events using DPDK rte_rtdsc > - No dependency on another library. Clean room native implementation of C= TF. > > Functional test case: > a) echo "trace_autotest" | sudo ./build/app/test/dpdk-test -c 0x3 --trac= e=3D.* > > The above command emits the following trace events > > uint8_t i; > > rte_trace_lib_eal_generic_void(); > rte_trace_lib_eal_generic_u64(0x10000000000000); > rte_trace_lib_eal_generic_u32(0x10000000); > rte_trace_lib_eal_generic_u16(0xffee); > rte_trace_lib_eal_generic_u8(0xc); > rte_trace_lib_eal_generic_i64(-1234); > rte_trace_lib_eal_generic_i32(-1234567); > rte_trace_lib_eal_generic_i16(12); > rte_trace_lib_eal_generic_i8(-3); > rte_trace_lib_eal_generic_string("my string"); > rte_trace_lib_eal_generic_function(__func__); > > > > Install babeltrace package in Linux and point the generated trace file > to babel trace. By default trace file created under > /dpdk-traces/time_stamp/ > > example: > # babeltrace /root/dpdk-traces/rte-2020-02-15-PM-02-56-51 | more > > [13:27:36.138468807] (+?.?????????) lib.eal.generic.void: { cpu_id =3D0, = name =3D "dpdk-test" }, { } > [13:27:36.138468851] (+0.000000044) lib.eal.generic.u64: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D 4503599627370496 } > [13:27:36.138468860] (+0.000000009) lib.eal.generic.u32: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D 268435456 } > [13:27:36.138468934] (+0.000000074) lib.eal.generic.u16: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D 65518 } > [13:27:36.138468949] (+0.000000015) lib.eal.generic.u8: { cpu_id =3D 0, n= ame =3D "dpdk-test" }, { in =3D 12 } > [13:27:36.138468956] (+0.000000007) lib.eal.generic.i64: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D -1234 } > [13:27:36.138468963] (+0.000000007) lib.eal.generic.i32: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D -1234567 } > [13:27:36.138469024] (+0.000000061) lib.eal.generic.i16: { cpu_id =3D 0, = name =3D "dpdk-test" }, { in =3D 12 } > [13:27:36.138469044] (+0.000000020) lib.eal.generic.i8: { cpu_id =3D 0, n= ame =3D "dpdk-test" }, { in =3D -3 } > [13:27:36.138469051] (+0.000000007) lib.eal.generic.string: { cpu_id =3D = 0, name =3D "dpdk-test" }, { str =3D "my string" } > [13:27:36.138469203] (+0.000000152) lib.eal.generic.func: { cpu_id =3D 0,= name =3D "dpdk-test" }, { func =3D "test_trace_points" } > > # There is a GUI based trace viewer available in Windows, Linux and Mac= . > It is called as tracecompass.(https://www.eclipse.org/tracecompass/) > > The example screenshot and Histogram of above DPDK trace using > Tracecompass. > > https://github.com/jerinjacobk/share/blob/master/dpdk_trace.JPG This series is quite big and did not get a lot of comments/reviews: especially the tracepoints added to important subsystems. - fixed some typos, some missed renames in commit logs and intermediate patches, some nits on coding style and reworded comments. - added the unit tests in MAINTAINERS. - moved the trace documentation a bit earlier in the documentation index (with EAL and core libraries). - had a go with test-null.sh with traces off/on (and enabling fp traces too), but did not do a good check on the performance impact. - the series compile on all my targets (and I did a per patch check), This new framework is promising but still experimental, what is important now is getting feedback. Thanks a lot for this work. For the series: Acked-by: David Marchand Series applied. --=20 David Marchand