From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0470A052E; Mon, 3 Feb 2020 14:32:06 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 10FFB1BFB5; Mon, 3 Feb 2020 14:32:06 +0100 (CET) Received: from inbox.dpdk.org (xvm-172-178.dc0.ghst.net [95.142.172.178]) by dpdk.org (Postfix) with ESMTP id 080891BFB1 for ; Mon, 3 Feb 2020 14:32:05 +0100 (CET) Received: by inbox.dpdk.org (Postfix, from userid 33) id D186FA0530; Mon, 3 Feb 2020 14:32:04 +0100 (CET) From: bugzilla@dpdk.org To: dev@dpdk.org Date: Mon, 03 Feb 2020 13:32:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: core X-Bugzilla-Version: 18.11 X-Bugzilla-Keywords: X-Bugzilla-Severity: critical X-Bugzilla-Who: scott_wasson@affirmednetworks.com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 Subject: [dpdk-dev] [Bug 389] Crash in librte_kni driver due to noncontiguous pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" https://bugs.dpdk.org/show_bug.cgi?id=3D389 Bug ID: 389 Summary: Crash in librte_kni driver due to noncontiguous pages Product: DPDK Version: 18.11 Hardware: All OS: All Status: UNCONFIRMED Severity: critical Priority: Normal Component: core Assignee: dev@dpdk.org Reporter: scott_wasson@affirmednetworks.com Target Milestone: --- We=E2=80=99re seeing a continuous crash since upgrading to 18.11, the kni F= IFO=E2=80=99s apparently aren=E2=80=99t contiguous. From user-space=E2=80=99s perspectiv= e, the kni=E2=80=99s tx_q straddles the 2MB page boundary at 0x17a600000. The mbuf pointers in the r= ing prior to this address are valid. The tx_q=E2=80=99s write pointer is indic= ating there are mbufs at 0x17a600000 and beyond, but the pointers are all NULL. Because the rte_kni kernel module is loaded: In eal.c: /* Workaround for KNI which requires physical address to work */ if (iova_mode =3D=3D RTE_IOVA_VA && rte_eal_check_module("rte_kni") =3D=3D 1) { if (phys_addrs) { iova_mode =3D RTE_IOVA_PA; Iova_mode is automatically forced to PA. We determined that enabling --legacy-mem caused the problem to go away. But this caused the locations of the kni=E2=80=99s data structures to move, so = they no longer straddled a hugepages boundary. Our concern is that the furniture m= ay move around again and bring us back to where we were. Being tied to using --legacy-mem is undesirable in the long-term, anyway. We also found that the following code patch helps (even without --legacy-me= m): index 3d2ffb2..5cc9d69 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -143,31 +143,31 @@ kni_reserve_mz(struct rte_kni *kni) char mz_name[RTE_MEMZONE_NAMESIZE]; snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_TX_Q_MZ_NAME_FMT, kni->name); - kni->m_tx_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_tx_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_tx_q =3D=3D NULL, tx_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_RX_Q_MZ_NAME_FMT, kni->name); - kni->m_rx_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_rx_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_rx_q =3D=3D NULL, rx_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_ALLOC_Q_MZ_NAME_FMT, kni->name); - kni->m_alloc_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_alloc_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_alloc_q =3D=3D NULL, alloc_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_FREE_Q_MZ_NAME_FMT, kni->name); - kni->m_free_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_free_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_free_q =3D=3D NULL, free_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_REQ_Q_MZ_NAME_FMT, kni->name); - kni->m_req_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_req_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_req_q =3D=3D NULL, req_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_RESP_Q_MZ_NAME_FMT, kni->name); - kni->m_resp_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_resp_q =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_resp_q =3D=3D NULL, resp_q_fail); snprintf(mz_name, RTE_MEMZONE_NAMESIZE, KNI_SYNC_ADDR_MZ_NAME_FMT, kni->name); - kni->m_sync_addr =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, 0); + kni->m_sync_addr =3D rte_memzone_reserve(mz_name, KNI_FIFO_SIZE, SOCKET_ID_ANY, RTE_MEMZONE_IOVA_CONTIG); KNI_MEM_CHECK(kni->m_sync_addr =3D=3D NULL, sync_addr_fail); return 0; I removed --legacy-mem, the tx_q still straddles the same 2MB page boundary, yet now everything seems OK. This would seem to follow precedent in rte_mempool.c: /* if we're trying to reserve contiguous memory, add appropriate * memzone flag. */ if (try_contig) flags |=3D RTE_MEMZONE_IOVA_CONTIG; which I think explains why our mbufs haven=E2=80=99t seen data truncation i= ssues. Could you please why RTE_MEMZONE_IOVA_CONTIG is necessary in PA mode? Isn= =E2=80=99t contiguousness a fundamental property of physical addressing? Are we still potentially vulnerable with --legacy-mem and without the above code change? Did we just get lucky because the furniture moved and doesn= =E2=80=99t straddle a page boundary at the moment? We also tested with 19.11 and did not see the crash. However the 19.11 rel= ease notes say: +* Changed mempool allocation behavior. Changed the mempool allocation behaviour so that objects no longer cross pa= ges by default. Note, this may consume more memory when using small memory page= s. Thanks! -Scott --=20 You are receiving this mail because: You are the assignee for the bug.=