From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C5FAA0A0E; Wed, 28 Apr 2021 09:47:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5583340147; Wed, 28 Apr 2021 09:47:02 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by mails.dpdk.org (Postfix) with ESMTP id 28BF640041 for ; Wed, 28 Apr 2021 09:47:00 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20210428074659euoutp01a84832062735f1eb1633e3c0c826789f~59f_-S_vw0120701207euoutp01e for ; Wed, 28 Apr 2021 07:46:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20210428074659euoutp01a84832062735f1eb1633e3c0c826789f~59f_-S_vw0120701207euoutp01e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1619596019; bh=cyjEcAPHa2F7wP8UXoIwaVnELFGdFbXVnZSdRpFP++I=; h=Subject:To:Cc:From:Date:In-Reply-To:References:From; b=Qbzf+23rkGNQ6/GpwRFtf5oNjiZuCTMpDmtSIaUylyhlMujL+WGpnR8stY2Cg9mnw /tehwXEYM8nOVhXqRxws1lFHK/QY28xyU9dWs4MUrJ66orOmglxbbfnv+YcdcnBHwD 6h6a17zesIwScmOm+RrRH0YLwrd+dD0BOr2gJxZ8= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20210428074659eucas1p19e262d077d163784ebec40dfc4cffb36~59f_1e3t22565225652eucas1p1m; Wed, 28 Apr 2021 07:46:59 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id DB.6C.09439.3F219806; Wed, 28 Apr 2021 08:46:59 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20210428074659eucas1p172dee99e02af729c1d4a04c1015b04e9~59f_ho_UY0219502195eucas1p1G; Wed, 28 Apr 2021 07:46:59 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210428074659eusmtrp24ef7518a87600e4761b938ca4f1ad703~59f_gwA6y1443014430eusmtrp2Q; Wed, 28 Apr 2021 07:46:59 +0000 (GMT) X-AuditID: cbfec7f5-c1bff700000024df-e3-608912f3999e Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id BD.59.08696.3F219806; Wed, 28 Apr 2021 08:46:59 +0100 (BST) Received: from [106.210.134.142] (unknown [106.210.134.142]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20210428074658eusmtip1cc4e4cb7c77f698a19f527bd9def20e9~59f98riAC2035920359eusmtip13; Wed, 28 Apr 2021 07:46:58 +0000 (GMT) To: Stanislaw Kardach , David Hunt , dev@dpdk.org Cc: upstream@semihalf.com, David Marchand From: Lukasz Wojciechowski Message-ID: <28f0ef16-4473-2151-928f-e66dd7b7c4aa@partner.samsung.com> Date: Wed, 28 Apr 2021 09:46:57 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.1 MIME-Version: 1.0 In-Reply-To: <20210426163310.1043438-1-kda@semihalf.com> Content-Transfer-Encoding: 8bit Content-Language: en-US X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupjleLIzCtJLcpLzFFi42LZduzned3PQp0JBi1NShZ9kz4yWWxf0cVm 8e7TdiaLs5PfMVo0Nx9kd2D1+LVgKavH4j0vmTze77vK5vFyS0EASxSXTUpqTmZZapG+XQJX xpOTs5gK3stU3Fo7nbGB8Yh4FyMnh4SAicTUfXfZuhi5OIQEVjBK7J79lwXC+cIo8fXKTDaQ KiGBz4wSkx/7wXTMObyZCaJoOaNEz7bnzBDOR0aJHa/Ws3cxcnAIC3hKvHnPCtIgIhAtcWTh LUYQm1nAVeL2j/NMIDabgK3EkZlfwWp4BdwkLn74DBZnEVCVmPtlD1i9qECSxM1L/5khagQl Ts58wgJicwpYSJzYfIAJYqa8RPPW2cwQtrjErSfzwY6TEDjBITH/+w1miKtdJB703WSBsIUl Xh3fwg5hy0j83wnTsI1R4urvn4wQzn5Gieu9K6CqrCUO//vNBvIZs4CmxPpd+hBhR4mLE+4x g4QlBPgkbrwVhDiCT2LStulQYV6JjjYhiGo9iac9Uxlh1v5Z+4RlAqPSLCSvzULyziwk78xC 2LuAkWUVo3hqaXFuemqxcV5quV5xYm5xaV66XnJ+7iZGYJI5/e/41x2MK1591DvEyMTBeIhR goNZSYSXbVdrghBvSmJlVWpRfnxRaU5q8SFGaQ4WJXHeXVvXxAsJpCeWpGanphakFsFkmTg4 pRqYJl4sDbzhv2fjoWu1fDYNF7z1Z3ueWO0qtZ3PZlqR9BwVudfV20pOLjzeEPryZ8yuXSkP 10axGExlkS++l/7jSVGvzeuUC6fCeRZbnxbx5uyWqHC1DjGPfvt8ZtS/gH0rQ1SVzhZbRX/y /brOW0gz60+u0777oRMc4jzqN02ammK0xK9A5aH3Ej/eqUtLnwYq7FfM/MazMnkK+z3rr7v6 kgyT5so99ZW4ob9JNuqE9qIlBVXsS9te3f6kwuvmdbzFobDzpns2i7mGhKO7uKGgX925HTEe Qd2nF2/ZeKH3EO/s/mTbNwYLL86yy2i7MfPJo0Uhu3s25x6dcmv3lDlPo/3aRfvnsDofclgX f6pZiaU4I9FQi7moOBEAaMoSn6EDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrAIsWRmVeSWpSXmKPExsVy+t/xu7qfhToTDI60c1v0TfrIZLF9RReb xbtP25kszk5+x2jR3HyQ3YHV49eCpawei/e8ZPJ4v+8qm8fLLQUBLFF6NkX5pSWpChn5xSW2 StGGFkZ6hpYWekYmlnqGxuaxVkamSvp2NimpOZllqUX6dgl6GU9OzmIqeC9TcWvtdMYGxiPi XYycHBICJhJzDm9m6mLk4hASWMoosfpyB2sXIwdQQkbiwyUBiBphiT/Xutggat4zSqxrb2QC qREW8JR4854VpEZEIFpi1485bCA2s4CrxO0f56FmdjNK/FrezwySYBOwlTgy8ytYA6+Am8TF D5+ZQGwWAVWJuV/2MILYogJJEvcur2SGqBGUODnzCQuIzSlgIXFi8wEmiAVmEvM2P2SGsOUl mrfOhrLFJW49mc80gVFoFpL2WUhaZiFpmYWkZQEjyypGkdTS4tz03GIjveLE3OLSvHS95Pzc TYzAqNp27OeWHYwrX33UO8TIxMF4iFGCg1lJhJdtV2uCEG9KYmVValF+fFFpTmrxIUZToH8m MkuJJucD4zqvJN7QzMDU0MTM0sDU0sxYSZzX5MiaeCGB9MSS1OzU1ILUIpg+Jg5OqQam1WL9 9smOvZNKD/h87eI4vTX+07KCEwutDF5abWvmXP5I1+rp5UPPd9m2fPlr9MNw5qkMce8nRz+U Pku469wipnjX7/8S60t6Zx7Mscm0PTn1ONffa0IbG3nv5vOWCExLKr0439dvS+NKEX6rbRUv 9FMKdstVipt/uVGTfrFHXzM55IwTr8WLPbx/k2ZG9r/PsE7rU5us+oGV59+fHDnj8DIPNol5 nW+eGm1yZ9ykVpLgty+w4IyHo1JxqM8t17pn13+d/HH8INeUuvI5q4P9Ep5P3y3yf7pNdcy3 6eklp149mljqaKJkGs/2zpbB9OPkXTOSJh7qP2G0azZX3r+HYZNEctaLW/5eufvVspcMSizF GYmGWsxFxYkA69bOXjMDAAA= X-CMS-MailID: 20210428074659eucas1p172dee99e02af729c1d4a04c1015b04e9 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20210426163404eucas1p13be6ae1bbcc4b947599cb606befd5370 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20210426163404eucas1p13be6ae1bbcc4b947599cb606befd5370 References: <20210426163310.1043438-1-kda@semihalf.com> Subject: Re: [dpdk-dev] [RFC PATCH] test/distributor: fix burst flush on worker quit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Stanislaw, W dniu 26.04.2021 o 18:33, Stanislaw Kardach pisze: > While working on RISC-V port I have encountered a situation where worker > threads get stuck in the rte_distributor_return_pkt() function in the > burst test. > After investigation some of the threads enter this function with > flag RTE_DISTRIB_GET_BUF set in the d->retptr64[0]. At the same time > main thread has already passed rte_distributor_process() so nobody will > clear this flag and hence workers can't return. > > What I've noticed is that adding a flush just after the last _process(), > similarly to how quit_workers() function is written in the > test_distributor.c fixes the issue. > Additionally the issue disappears when I remove the rdtsc delay code > inside the rte_distributor_request_pkt(). > However I can't get this to reproduce on x86 (even with SIMD forced > off) and with artificial delays, which is why I wonder whether I'm not > actually hiding some other issue. I was able to reproduce the issue on x86 arch using VM with 32 emulated CPU cores. I guess it would be enough to just have more than 8 worker threads, so not all of them would be awaken. > > Looking at the implementation of the distributor, it is based on > __atomic_* builtins and the only platform related bit in the fast-path > is the rte_rdtsc() and rte_pause(). There may be some issues in the > toolchain (I've tried so far with the Ubuntu one - 10.2.0-8ubuntu1). > I should add that all unit tests for distributor are passing so either > there's some coverage corner case or the implementation works on RISC-V. > As for RDTSC I'm using a sleep-stable time counter with 1MHz frequency > and switching to high resolution cycle counter also removes the issue > but that's the same as removing the rdtsc delay as mentioned above. > > I'd love to hear from You if this fix makes any sense. Yes your patch fixes the issue and is perfectly fine. > > While modifying this test, I've also pulled in a fix from > test_distributor.c which ensures that each thread gets his own wakeup > packet as it's possible that when sending a burst of packets, they won't > be spread over all the workers. > > Signed-off-by: Stanislaw Kardach > Fixes: 7c3287a10535 ("test/distributor: add performance test for burst mode") > Cc: david.hunt@intel.com > Cc: l.wojciechow@partner.samsung.com > Cc: David Marchand > --- > app/test/test_distributor_perf.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c > index b25f79a34..fdbeae6d2 100644 > --- a/app/test/test_distributor_perf.c > +++ b/app/test/test_distributor_perf.c > @@ -188,13 +188,15 @@ quit_workers(struct rte_distributor *d, struct rte_mempool *p) > rte_mempool_get_bulk(p, (void *)bufs, num_workers); > > quit = 1; > - for (i = 0; i < num_workers; i++) > + for (i = 0; i < num_workers; i++) { > bufs[i]->hash.usr = i << 1; > - rte_distributor_process(d, bufs, num_workers); > + rte_distributor_process(d, &bufs[i], 1); > + } > > rte_mempool_put_bulk(p, (void *)bufs, num_workers); > > rte_distributor_process(d, NULL, 0); > + rte_distributor_flush(d); > rte_eal_mp_wait_lcore(); > quit = 0; > worker_idx = 0;     Tested-by: Lukasz Wojciechowski     Reviewed-by: Lukasz Wojciechowski -- Lukasz Wojciechowski Principal Software Engineer Samsung R&D Institute Poland Samsung Electronics Office +48 22 377 88 25 l.wojciechow@partner.samsung.com