From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB15343866; Mon, 8 Jan 2024 16:13:29 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 556504027E; Mon, 8 Jan 2024 16:13:29 +0100 (CET) Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by mails.dpdk.org (Postfix) with ESMTP id 7229740261 for ; Mon, 8 Jan 2024 16:13:27 +0100 (CET) Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4T7yFp5mTlz6FGVm; Mon, 8 Jan 2024 23:11:42 +0800 (CST) Received: from frapeml100007.china.huawei.com (unknown [7.182.85.133]) by mail.maildlp.com (Postfix) with ESMTPS id 466F9140B33; Mon, 8 Jan 2024 23:13:26 +0800 (CST) Received: from frapeml500007.china.huawei.com (7.182.85.172) by frapeml100007.china.huawei.com (7.182.85.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 8 Jan 2024 16:13:26 +0100 Received: from frapeml500007.china.huawei.com ([7.182.85.172]) by frapeml500007.china.huawei.com ([7.182.85.172]) with mapi id 15.01.2507.035; Mon, 8 Jan 2024 16:13:26 +0100 From: Konstantin Ananyev To: Stephen Hemminger , "dev@dpdk.org" CC: "arshdeep.kaur@intel.com" , "Gowda, Sandesh" , Reshma Pattan Subject: RE: Issues around packet capture when secondary process is doing rx/tx Thread-Topic: Issues around packet capture when secondary process is doing rx/tx Thread-Index: AQHaQdZbww5LrB7u+kG/lMknfiGZILDP/kIw Date: Mon, 8 Jan 2024 15:13:25 +0000 Message-ID: <5c28d2a26f5142c3a509cc8bda2fca75@huawei.com> References: <20240107175900.1276c0a5@hermes.local> In-Reply-To: <20240107175900.1276c0a5@hermes.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.206.138.42] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > I have been looking at a problem reported by Sandesh > where packet capture does not work if rx/tx burst is done in secondary pr= ocess. >=20 > The root cause is that existing rx/tx callback model just doesn't work > unless the process doing the rx/tx burst calls is the same one that > registered the callbacks. >=20 > An example sequence would be: > 1. dumpcap (or pdump) as secondary tells pdump in primary to register ca= llback > 2. secondary process calls rx_burst. > 3. rx_burst sees the callback but it has pointer pdump_rx which is not n= ecessarily > at same location in primary and secondary process. > 4. indirect function call in secondary to bad location likely causes cra= sh. As I remember, RX/TX callbacks were never intended to work over multiple pr= ocesses. Right now RX/TX callbacks are private for the process, different process si= mply should not see/execute them. I.E. it callbacks list is part of 'struct rte_eth_dev' itself, not the rte_= eth_dev.data that is shared between processes. It should be normal, wehn for the same port/queue you will end-up with diff= erent list of callbacks for different processes. =20 So, unless I am missing something, I don't see how we can end-up with 3) an= d 4) from above: >From my understanding secondary process will never see/call primary's callb= acks. About pdump itself, it was a while when I looked at it last time, but as I = remember to start it to work, server process has to call rte_pdump_init() which in terns register PDUMP_M= P handler. I suppose for the secondary process to act as a 'pdump server' it needs to = call rte_pdump_init() itself, though I am not sure such option is supported right now.=20 =20 >=20 > Some possible workarounds. > 1. Keep callback list per-process: messy, but won't crash. Capture won't= work > without other changes. In this primary would register callback= , but secondaries > would not use them in rx/tx burst. >=20 > 2. Replace use of rx/tx callback in pdump with change to rte_ethdev to h= ave > a capture flag. (i.e. don't use indirection). Likely ABI prob= lems. > Basically, ignore the rx/tx callback mechanism. This is my pre= ferred > solution. It is not only the capture flag, it is also what to do with the captured pa= ckets (copy? If yes, then where to? examine? drop?, do something else?). It is probably not the best choice to add all these things into ethdev API. > 3. Some fix up mechanism (in EAL mp support?) to have each process fixup > its callback mechanism. =20 Probably the easiest way to fix that - pass to rte_pdump_enable() extra inf= ormation that would allow it to distinguish on what exact process (local, remote) we want to enable pdump functionality. Then it could act accordingly. >=20 > 4. Do something in pdump_init to register the callback in same process c= ontext > (probably need callbacks to be per-process). Would mean callback is a= lways > on independent of capture being enabled. >=20 > 5. Get rid of indirect function call pointer, and replace it by i= ndex into > a static table of callback functions. Every process would have= same code > (in this case pdump_rx) but at different address. Requires al= l callbacks > to be statically defined at build time. Doesn't look like a good approach - it will break many things.=20 =20 > The existing rx/tx callback is not safe id rx/tx burst is called from dif= ferent process > than where callback is registered. =20