From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A493CA0507; Fri, 1 Apr 2022 12:49:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6366042911; Fri, 1 Apr 2022 12:49:43 +0200 (CEST) Received: from inbox.dpdk.org (inbox.dpdk.org [95.142.172.178]) by mails.dpdk.org (Postfix) with ESMTP id 89EAD4067E for ; Fri, 1 Apr 2022 12:49:42 +0200 (CEST) Received: by inbox.dpdk.org (Postfix, from userid 33) id 663FBA0508; Fri, 1 Apr 2022 12:49:42 +0200 (CEST) From: bugzilla@dpdk.org To: dev@dpdk.org Subject: [Bug 990] mlx5 pmd crashing when trying to free mbuf from secondary process when mprq is enabled Date: Fri, 01 Apr 2022 10:49:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: ethdev X-Bugzilla-Version: 21.11 X-Bugzilla-Keywords: X-Bugzilla-Severity: critical X-Bugzilla-Who: sahithi.singam@oracle.com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org https://bugs.dpdk.org/show_bug.cgi?id=3D990 Bug ID: 990 Summary: mlx5 pmd crashing when trying to free mbuf from secondary process when mprq is enabled Product: DPDK Version: 21.11 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: critical Priority: Normal Component: ethdev Assignee: dev@dpdk.org Reporter: sahithi.singam@oracle.com Target Milestone: --- This issue is always reproducible when any two DPDK applications (primary , secondary) are run on mellanox virtual functions using mlx5_pmd with mprq enabled , segfault occurs whenever the secondary application tries to free a packet received by primary application. Packet size received should be grea= ter than 128Bytes to reproduce this issue.=20=20 Run multi process sample application as below on a node with mellanox virtu= al functions using mlx5_pmd.=20 First start server application as below 1. /opt/dpdk-mp_server -l 2-3 -n 4 --allow 0000:00:05.0,mprq_en=3D1,rxqs_min_mprq=3D1,mprq_log_stride_num=3D9 --allow= =20 0000:00:06.0,mprq_en=3D1,rxqs_min_mprq=3D1 ,mprq_log_stride_num=3D9 --proc-type=3Dprimary -- -p 0x3 -n 1=20 2. Once server is started ,start client application as below=20 ~ # /opt/dpdk-mp_client -l 4 -n 4 --proc-type=3Dauto -- -n 0 3. Then from a different machine connected to the above machine, try to send traffic to the above server/client application using testpmd run as below (= note pktsize should be greater than 128Bytes to reproduce this crash, I am using 256Bytes packets).=20 /opt/dpdk-testpmd -l 2-3 -m 4 --allow 0000:00:04.0,mprq_en=3D1,rxqs_min_mprq=3D1,mprq_log_stride_num=3D9 -- --po= rtmask=3D0x1 --mbcache=3D64 --forward-mode=3Dtxonly=20 --eth-peer=3D0,02:00:17:0A:4B:FB --stats-period=3D10 --txpkts=3D256 --tx-ip=3D10.1.12.253,10.1.12.27 --txq 1 --rxq 1=20 Once the traffic is received by server application, server will receive pac= kets in mbuf and will send it to client. client will try to free the mbuf during which it is giving segmentation fault.=20 Rootcause for this issue : =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D When server tries to receive the packet, mlx5_pmd is internally calling mprq_buf_to_pkt which will use and attach external buffer to the mbuf (as t= he packets from testpmd are greater than 128 bytes size). When external buffers are used, mlx5_pmd will internally sets the callback function, which should free this external buffer to the address of mlx5_mprq_buf_free_cb=20 i.e mbuf->shinfo->free_cb will be set to address of mlx5_mprq_buf_free_cb in primary application (i.e dpdk-mp_server). But when this mbuf is given to the secondary application i.e dpdk-mp_client= , while freeing this mbuf, secondary application will try to invoke free_cb in mbuf but as this address is not mapped correctly in secondary application, = it will give a segmentation fault or could lead to corruption.=20 [linux-shell]$ nm dpdk_build/examples/dpdk-mp_client |grep -i=20 "mlx5_mprq_buf_free_cb" 00000000008886e0 T mlx5_mprq_buf_free_cb [linux-shell]$ nm dpdk_build/examples/dpdk-mp_server |grep -i=20 "mlx5_mprq_buf_free_cb" 0000000000888e00 T mlx5_mprq_buf_free_cb --=20 You are receiving this mail because: You are the assignee for the bug.=