From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0042.outbound.protection.outlook.com [104.47.33.42]) by dpdk.org (Postfix) with ESMTP id 30A4A1B1A3 for ; Mon, 8 Jan 2018 16:32:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=1sY/vyE2K8m/sM0MTAw7MKDP8r9ZqzQOY48GG11/WsQ=; b=EjMpIPo0Wf4mCEedfN4qRf1UjVj27zrQGg5Zfvu6jnSo2+/ICMSSTtlAgSgNxLdg+CwrhZfXp9QKo21d1jnXL48vX2+Qhb6/8ZgoZZRk2vc90qQh4IwCwGa1dsvPQG7CsLNvYwxTe6sXGy3THRQ9hL5tvku8QWCLqNlQ4wU9Z4g= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from Pavan-LT (111.93.218.67) by CY4PR07MB3464.namprd07.prod.outlook.com (10.171.252.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Mon, 8 Jan 2018 15:32:39 +0000 Date: Mon, 8 Jan 2018 21:02:20 +0530 From: Pavan Nikhilesh To: Gage Eads , harry.van.haaren@intel.com, jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com Cc: dev@dpdk.org Message-ID: <20180108153219.jszoepdgfiggn3bm@Pavan-LT> References: <1512011314-19682-1-git-send-email-gage.eads@intel.com> <1512011314-19682-2-git-send-email-gage.eads@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1512011314-19682-2-git-send-email-gage.eads@intel.com> User-Agent: NeoMutt/20170609 (1.8.3) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: SG2PR06CA0130.apcprd06.prod.outlook.com (10.175.207.160) To CY4PR07MB3464.namprd07.prod.outlook.com (10.171.252.145) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7a1b1624-8db7-436e-297b-08d556ad0eaf X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:CY4PR07MB3464; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3464; 3:NkNvHf9qeQx5G5b7F0dnnB3HHLuBr+EOgpLMRO8ePejOBvDjYVtWheDdOK2iGJ2ClmTB52zjS5l/Mm9GaqxG3J9CuygHUtoo18q4Q2gjnbCP13SZ2njeHUAya+Us6cjWso7bA2JtYdHOnFegMVHkxc37vg8ZPx2TrPYHa1orVlV93n/1gtuAyE0p1RpNhA+nydIXGOmh0zsjol89Wmo6+QE9mztzrEa1IACS8VM5rmrzxxNeFRLTr+CZrw4g9E9p; 25:jG5A5pkGXTVHjBWNMxO6W9q8BrKDsWhn74SM1Qgkynb23brpudzYJzMROsxrFYmt2yHNemwJ0XpnzMspaDd6ZFzmwmsQUkqz6+sqQN2CA6pQEZK/3vgrdfL3+AVEVNs738xfwnQbyzboR2Mp9IWBVP3nQmE2Z/IHPag6u0JIFixG5It7vst2I+CPENAZ2443iQe7bgcFmmSapiURtzeuz63h9ZP6MNaWOyCw/SpccHiQgsrlxLWua3yhKh2/lz2gx8AAPS5+Q+LurJV3Qk5lpa10MR5BePgOC6Y3Dhu++NFMMlAvUyt0X6Ec4GTH9Eq7dtH5FXlUq6Lb1Dm2evNkvw==; 31:a9beuy1TAZJcW+ge+0MS9nSIdI5YkalKBqLRQvmCMQiAAEdyslKlow/0X9Q33IbrMSFOdPtsSab3sJ35uL88ixH1lRGvzB8eVOeS8GGvv1/znd+5i4zuvPtKma8uMUM/x94fLzimlDw8nLRXXMDHvK0dpm/a0g+j09+RFeMtj+jljGJlaFYhHOC6Vc4TqfSRMU3/+iwYBcZuSD2bIe9Y+qqrExwQhwQQAftCxk7UIrA= X-MS-TrafficTypeDiagnostic: CY4PR07MB3464: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3464; 20:kQgV7io2q8ACJyuaoSo4/8WK8Q0sGTA4TdTb0gIeLQQth1fh2sB+8n6Cf0TQ/XxmiUBFWugdH6TkC9MNYyRUmN4s0eDQoQ4LmTsYcmb/3qpNYfNWKxB8kytOnUgB4qO8kXGusdqlXSqS4/pb2p7ipSCRe4IqmZThb0834ET6qDU8a1kAKZjIzIaTF2LiYBGsDcQV27QDN+dWIpG9hd4scT5VvJcvPN+rQwpSsMitxxvZApcUyT63MXQSIOlJzgpMLIYBkb/pDVOFTgNVfHkkyAp1d9f+E7PBAcEcQ3w5TH7+oHSZrhJACKJmmea3WI+8+rVXYZP0Oinrvf6ChUzXaCgE3l9CZHydTw7C7PDs53CbFh1A+lFojKRhTmSdRo3BJkQidM4XLotZZpz5w/YL71Mxe+DEfwa8Rzmu0sgLHA0B098KcefpPZbEbcNIe76y+IKoNqu7RHqbkk2/Y3JeCZ44F1woAALhACdAAJ1w2hs4OqhfXHna6rhb2GIbVhj+lFyLGM2JbeT1FCiPKi5EpONUTPMjV8kn8Z/ixblaHTQP4I/4+OS0lAKhXZOTzwQ14L1AEVJisCL2t1fZrjCWnyuD8YyuISrRTUuAbV3MhhE=; 4:enKY68BVmyjNHZXEOgPAcb69w5WMskosHWXwW8608cj3cD1AV9i/VnEPPy7vGttU0ZWbG9J4nboKoW3N73tVfpmCWXJqIWwmEU6NuVzuGVcSHoZmbilOO+6MgKPX5AB6rnJGhmNtAymIojqf26aPwvOVoQaQ4LSMXJuo9Ub83uSaM3mpzN8PDKgpsQvWQwmRcdl6dP6rSvnFWuMKYoOzKLok4nzhFeosnCinq3Pp+eu0yz8GApGyfkdTBZ50A/vXgLL/TIkrawDIIB8AucfCZHLMCB6i1d3Q0hQYnQ7Miywx8cgkwgMRl8FwnRzGacU1 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(944501075)(3002001)(10201501046)(93006095)(6041268)(20161123560045)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011); SRVR:CY4PR07MB3464; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:CY4PR07MB3464; X-Forefront-PRVS: 054642504A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(346002)(396003)(376002)(366004)(39380400002)(39850400004)(24454002)(199004)(189003)(6306002)(33716001)(105586002)(575784001)(83506002)(9686003)(6246003)(7736002)(6636002)(8936002)(2950100002)(42882006)(305945005)(106356001)(6666003)(68736007)(3846002)(23726003)(50466002)(55016002)(81156014)(5660300001)(478600001)(4326008)(81166006)(53376002)(6116002)(5009440100003)(1076002)(72206003)(53936002)(97736004)(386003)(16526018)(2906002)(966005)(229853002)(59450400001)(8676002)(33896004)(316002)(25786009)(58126008)(52116002)(76176011)(6496006)(47776003)(66066001)(16586007)(107986001)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3464; H:Pavan-LT; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3464; 23:/qHORVtk0hGfrpgRAxaVKQLyGf1YF++lljzbULDMu?= =?us-ascii?Q?9Mstzzx0Iy6SfjvrvKWunQ63Q9aaEWioGySA22Ex+urvbH9y7ccc1n87wivn?= =?us-ascii?Q?LOkAxR1xg7jHT2xZ7WWFXMMRKNNMFQdHOXv0QVViz807Loomqkid7dfrh97y?= =?us-ascii?Q?/rcYVKZRq9Bv3jJNi+hzofX2qkZ4+SlHpKDa6Ed2M97Y+t92C7iNAbt+7HQC?= =?us-ascii?Q?S21/m1t+Mgj3AdyGSS1GC960lLxC4GAopnJdJxhCUzBvyos0bRWQr+HmF083?= =?us-ascii?Q?zY602pi5tkTPC9/Y0nVoX6VLb50f4NQmzq2pGb7Z0f35bkk7TWhQnEppOduj?= =?us-ascii?Q?SJzMJCeasZ5v4na61LWeB3CEUEGI+8fdoJenoZYOcKl7bigHrjimYiHRBnfH?= =?us-ascii?Q?KHhTgByf+K7VWy8gZ/3Ah+MsgKjX7tnkYCbarNBy7tvh95Y+g9OIUB/cZKQa?= =?us-ascii?Q?T516qSMup6joPFuIvp2b43sdyflMkkGCkjgbEk3Hll4qcu9qGpaLFLss3Ttu?= =?us-ascii?Q?U3WG/3FRPIThz+iPk17n7ousI3zuhtOrmt3I9MhdXpUzXeGQeCcAmYjJG/cZ?= =?us-ascii?Q?Oo2KqX4XreYRedVIg3PcO9Hv5HUkym+RLn1DglzJrlV87yuArjDPZUKt6l+x?= =?us-ascii?Q?Gf/0QT6/1SJofyVrTGuccwhyWT/TrLyQeVeYgmqUbD52axtuS5LcNYqlneId?= =?us-ascii?Q?2efz7Bbi0Yf8BhwzFtV3kwTqRUJr1DvyO3+caLyJgJjyi7vtt2TKn+2ZEXX/?= =?us-ascii?Q?1AurKc6d2MeoQGCwrcRpbCp12s7DgDbgFor2zobWgziZTQaLaT13D/wJuwc5?= =?us-ascii?Q?+WnLyti4kCEvnUcZ8K4jv4gq+Yfg4DzcZrHP9UMLP2YWUtG2qWp5oOMR1No6?= =?us-ascii?Q?2ZLiZgYIf3KJzXAUR2F5QeZIe70xq+ifeT2+HLVnOQhm9YqxtHq776UdC6AG?= =?us-ascii?Q?Sc2A5O14Ozt57EeWSdXQWrFZ3wbzQdsGl82q10qEgC0fMlfADBO3rcEGFV/S?= =?us-ascii?Q?c4Wp/Xgvw5FAE4R1V5IXsUbTMtuvNHDt7IGd/tqE+EpEHVIRGQUKwmYiaQQv?= =?us-ascii?Q?CwRqjFNQXEKKNYF0UQAulHZNx9b41N+RsQsMigKZsxqOPZGfa4VhY4gfXuyj?= =?us-ascii?Q?q9oJ91Sn5O0dO7ec2qUaXy5c0PJAohJIv0YLIw/AYrzZruscLR1ewxES2VNC?= =?us-ascii?Q?LYQ9NE7WiEPlCiviH7p6kivel+BeQT8iAAMUlF8hkhiUDVZo4gl0C23GaWH3?= =?us-ascii?Q?C5ITBrdx/O4F2imVOb7fGvJuGxbUSehHryahTDHmpdXMcubVdtV7YVtpaPBs?= =?us-ascii?Q?PX1fjwi8FZ50f9gQZm8/FhuWGjri4/xcbdx6Vvu798bLOt45/X0BF1I4AC7l?= =?us-ascii?Q?e6AX589b5V6yM35aL7s4J7iDZ8=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3464; 6:w5b4b4zUJ2uWdpqW9oNYK0zID1UxRhIYq6pS8WUdpoe3Denfm5VIvOms/6e/+6C3643vgcTlF2rCJW9oPvApMdSXdS41pfBtXSOjOw37lsXH5jt8f3jE2XVzTNNnSuOXLiP1+tH4SWFEaCLcR6EOI7vjbIcW62IWqNL3rupHnhsFJt/pNidmyAbmL+/mrOHDRSrmK88xtXsaaSt8dHZcMPhZtErLRl7GDVgGwdeU3tKjyoAu+M4Niw90uO04srk6cRDb/OCrmuJBhlSZaZL1Cj4F4fwAyCyN0XkclQ9NyPT+o7E+3itlXV3rQI5xtuPU6JUQQOwzC73x/pD0mwIrFqvCnPhYh40o2pAGL+75yJU=; 5:3wSNdP9B1jTK3rln74yjFa8nmoI4g9YDra0RiVr45gKmwtgc7oJYokJTS/vBvsiEs7G0bCCJXke1rlVBPOcseO50RaHfuWexRRZSHNaw2CYGLqC4gHHz0xZk6sdgzyYy7F94Uz5QfYxrVoU+BDEoMzHHaa1lBNGY+GMpH+new68=; 24:Q3MNHk2bPBai8v5DAa4ajS/+VvfFSB9aKl0/xbTP8q9g+MAC1D5WkE5BNbFwIDPZlJhdWg2OclnyqphJyPi4Xxg6AEbVLipYoCii4K7qKOo=; 7:Ezakd5p+gIAolpvSlWRIjdjTEUZN+wmQuD9H92KNpS2fyBAXknKZ/Xgb0DKFg+Ed0E62GWQWnsYRyFRshUBdIAcQfsc0uCCr1/kUaqnZyR3vWibEM0DatKYJN7r9Gy3mdXDEElZajuMybA1VAuCHFP6JuOcMXA2aV1ATXnizf3zhUXM1b5vbShfCUdh8W8+6ELH05gc50LWZ0hbslvbcj/TxeWTFjSr6jxK4iUmIMsXxTXnw7ngAVVlXnavf7wbb SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2018 15:32:39.8798 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7a1b1624-8db7-436e-297b-08d556ad0eaf X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3464 Subject: Re: [dpdk-dev] [PATCH 2/2] event/sw: use dynamically-sized IQs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jan 2018 15:32:43 -0000 On Wed, Nov 29, 2017 at 09:08:34PM -0600, Gage Eads wrote: > This commit introduces dynamically-sized IQs, by switching the underlying > data structure from a fixed-size ring to a linked list of queue 'chunks.' > This has a number of benefits: > - Certain corner cases were observed in which all of a pipeline's flows > could be pinned to one port for extended periods, effectively turning a > multi-core pipeline into single-core one. This was caused by an event > producer having a larger new_event_threshold than the IQ depth, and > injecting large numbers of packets that are ultimately backpressured in a > worker's rx_ring, causing those packets' flows to be scheduled to that > port. > The dynamically sized IQ does not have this problem because each IQ can > grow large enough to store all the system's events, such that > backpressure will not reach the worker_ring. > - Slight performance improvement (~1-2%) in high throughput scenarios, > tested with eventdev_pipeline_sw_pmd. > > This implementation has a small increase in the queue storage memory > footprint (~70KB). This commit also removes the iq_size xstat, which no > longer applies to this implementation. > > Signed-off-by: Gage Eads > --- Sw eventdev crashes when used alongside Rx adapter. The crash happens when pumping traffic at > 1.4mpps. This commit seems responsible for this. Apply the following Rx adapter patch http://dpdk.org/dev/patchwork/patch/31977/ Command used: ./build/eventdev_pipeline_sw_pmd -c 0xfffff8 --vdev="event_sw" -- -r0x800 -t0x100 -w F000 -e 0x10 Backtrace: Thread 4 "lcore-slave-4" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xffffb6c8f040 (LWP 25291)] 0x0000aaaaaadcc0d4 in iq_dequeue_burst (count=48, ev=0xffffb6c8dd38, iq=0xffff9f764720, sw=0xffff9f332600) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/iq_chunk.h:142 142 ev[total++] = current->events[index++]; (gdb) bt #0 0x0000aaaaaadcc0d4 in iq_dequeue_burst (count=48, ev=0xffffb6c8dd38, iq=0xffff9f764720, sw=0xffff9f332600) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/iq_chunk.h:142 #1 sw_schedule_atomic_to_cq (sw=0xffff9f332600, qid=0xffff9f764700, iq_num=0, count=48) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/sw_evdev_scheduler.c:74 #2 0x0000aaaaaadcdc44 in sw_schedule_qid_to_cq (sw=0xffff9f332600) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/sw_evdev_scheduler.c:262 #3 0x0000aaaaaadd069c in sw_event_schedule (dev=0xaaaaaafbd200 ) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/sw_evdev_scheduler.c:564 #4 0x0000aaaaaadca008 in sw_sched_service_func (args=0xaaaaaafbd200 ) at /root/clean/rebase/dpdk-next-eventdev/drivers/event/sw/sw_evdev.c:767 #5 0x0000aaaaaab54740 in rte_service_runner_do_callback (s=0xffff9fffdf80, cs=0xffff9ffef900, service_idx=0) at /root/clean/rebase/dpdk-next-eventdev/lib/librte_eal/common/rte_service.c:349 #6 0x0000aaaaaab54868 in service_run (i=0, cs=0xffff9ffef900, service_mask=18446744073709551615) at /root/clean/rebase/dpdk-next-eventdev/lib/librte_eal/common/rte_service.c:376 #7 0x0000aaaaaab54954 in rte_service_run_iter_on_app_lcore (id=0, serialize_mt_unsafe=1) at /root/clean/rebase/dpdk-next-eventdev/lib/librte_eal/common/rte_service.c:405 #8 0x0000aaaaaaaef04c in schedule_devices (lcore_id=4) at /root/clean/rebase/dpdk-next-eventdev/examples/eventdev_pipeline_sw_pmd/main.c:223 #9 0x0000aaaaaaaef234 in worker (arg=0xffff9f331d80) at /root/clean/rebase/dpdk-next-eventdev/examples/eventdev_pipeline_sw_pmd/main.c:274 #10 0x0000aaaaaab4382c in eal_thread_loop (arg=0x0) at /root/clean/rebase/dpdk-next-eventdev/lib/librte_eal/linuxapp/eal/eal_thread.c:182 #11 0x0000ffffb7e46d64 in start_thread () from /usr/lib/libpthread.so.0 #12 0x0000ffffb7da8bbc in thread_start () from /usr/lib/libc.so.6 Segfault seems to happen in sw_event_schedule and only happens under high traffic load. Thanks, Pavan > drivers/event/sw/iq_chunk.h | 212 ++++++++++++++++++++++++++++++++++ > drivers/event/sw/iq_ring.h | 172 --------------------------- > drivers/event/sw/sw_evdev.c | 59 ++++++---- > drivers/event/sw/sw_evdev.h | 16 ++- > drivers/event/sw/sw_evdev_scheduler.c | 40 +++---- > drivers/event/sw/sw_evdev_xstats.c | 12 +- > test/test/test_eventdev_sw.c | 15 +-- > 7 files changed, 288 insertions(+), 238 deletions(-) > create mode 100644 drivers/event/sw/iq_chunk.h > delete mode 100644 drivers/event/sw/iq_ring.h > > @@ -1743,7 +1739,6 @@ xstats_id_reset_tests(struct test *t) > 0, /* tx */ > 0, /* drop */ > 7, /* inflight */ > - 512, /* iq size */ > 0, /* iq 0 used */ > 0, /* iq 1 used */ > 0, /* iq 2 used */ > -- > 2.7.4 >