From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9C7EA2EEB for ; Tue, 10 Sep 2019 21:20:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBB741EB89; Tue, 10 Sep 2019 21:20:43 +0200 (CEST) Received: from mails1n0-route0.email.arizona.edu (mails1n0-route0.email.arizona.edu [128.196.130.69]) by dpdk.org (Postfix) with ESMTP id B618B1EB4E for ; Tue, 10 Sep 2019 21:20:41 +0200 (CEST) IronPort-SDR: oIL497NevVt22rMNBipMB9oVTGF7QGC4YEOfzZU3oBGUVfXy76OIH0cpEVzOFPg06xnTf5o4Uo E+habBG1sEeA== IronPort-PHdr: =?us-ascii?q?9a23=3AUJSz3hddpGwFywf/WiuuRwQTlGMj4u6mDksu8p?= =?us-ascii?q?Mizoh2WeGdxcS/bB7h7PlgxGXEQZ/co6odzbaP6Oa4ASdZuMzJ8ChbNsAVDV?= =?us-ascii?q?ld0YRetjdjKfbNMVf8Iv/uYn5yN+V5f3ghwUuGN1NIEt31fVzYry76xzcTHh?= =?us-ascii?q?LiKVg9fbytScbdgMutyu+95YDYbRlWizqhe7NyKwi9oRnMusUMjoZuN6g8xg?= =?us-ascii?q?HVrndUdOha2H5kKUyOlBr4+su84YRv/itNt/4/7cJMTbn2c6ElRrFEEToqNH?= =?us-ascii?q?w46tf2vhfZVwuP4XUcUmQSkhVWBgXO8Q/3UJTsvCbkr+RxwCaVM9H4QrAyQj?= =?us-ascii?q?Si8rxkSAT0hycdNj4263/Yh8pth69Guh2hphh/w4nJYIGJMfd1Y63Qcc8GSW?= =?us-ascii?q?dHQ81cUTFKDIGhYIsVF+cPPehWoZTjqVYVsReyGRWgCP/zxjNUmnP727Ax3e?= =?us-ascii?q?Q7EQHB2QwtB9YAsGnSrNT6NacdS+e1x7TVwzXDbvNdxDDw6IfJchA9uvyMWK?= =?us-ascii?q?h/ccvVyUUyDQ/EjlKQqZH/PzOOy+QNtW6b7up6We2zjG4nrhh8rz6yzckijY?= =?us-ascii?q?nJg5gaylHC9Shh3Yk1PcC3SE1hbd6jCptQuCeXPJZ1TMM6W2xkpSI3xqMctZ?= =?us-ascii?q?KmYiQG0pUqyhrFZ/GHa4SE/xbuWPiLLTtlmn5pYrOyihKo/US9yeDxV8+520?= =?us-ascii?q?tQoCVfiNnDrHUN2gTW6siAV/Ry4F+s2S2K1wDP8uFEJl00lbbDJ54h3LEwkp?= =?us-ascii?q?0TvFzEHi/uhEX2jaqWeV8+9ue08OTreK/ppoWdN451jQH+NL4hltahDuslLw?= =?us-ascii?q?gCR2mb+eKi273/5UD1XbFHguMsnqXErpzWP8UWq6qjDwJVyooj7gywDzai0N?= =?us-ascii?q?QWh3kHK1dFdQqcj4jpOlHOIu34DfGwglStijhrwOvLMaH6ApnXNHfDirbhfL?= =?us-ascii?q?d6605a0gU/185Q6I9JCr0ZOvL8RlfxtMDEDh8+KwG0zOfnCMlm24MbQ22PA6?= =?us-ascii?q?uZPL7IvV6I4eIvPvOMZJQIuDb7Lfgk5+TigmM+mV8YZaOpx4cYaGikHvR6JE?= =?us-ascii?q?WUeWfjjckfEWoRowU+VPHliF6YXT5UfXayUPF02jZuBI+4JYvbSoPrh6aOjw?= =?us-ascii?q?mhGZgDRH1AGlGFFj/Td4TMD/gNdAqXK85ly3oVUqW7UJIn0wDovwj/nek0Zt?= =?us-ascii?q?HI8zEV4MqwnON+4PfewEk/?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2C4AwDa9nddjAVFDMBlHQEBBQEHBQG?= =?us-ascii?q?BZ4MFUzIqjR2GFgFTBoQ+AYZqkS8BAQEMAQETEAkBAgEBhwojOBMCAwkBAQE?= =?us-ascii?q?EAQEBAQEGAgIBEwEBASaFXgyCOiKDHIEONGGDIgGBdhQPmUmPfjOKOwaBNIt?= =?us-ascii?q?4GHiBB4RhgmEEGIcqBKxHgiuHAY13GpkJAZYDkGoCBAYFAhQBgWkggVkzGgg?= =?us-ascii?q?bFYMngk4XiGOFP0ABMgGBBQEBjyABAQ?= X-IPAS-Result: =?us-ascii?q?A2C4AwDa9nddjAVFDMBlHQEBBQEHBQGBZ4MFUzIqjR2GF?= =?us-ascii?q?gFTBoQ+AYZqkS8BAQEMAQETEAkBAgEBhwojOBMCAwkBAQEEAQEBAQEGAgIBE?= =?us-ascii?q?wEBASaFXgyCOiKDHIEONGGDIgGBdhQPmUmPfjOKOwaBNIt4GHiBB4RhgmEEG?= =?us-ascii?q?IcqBKxHgiuHAY13GpkJAZYDkGoCBAYFAhQBgWkggVkzGggbFYMngk4XiGOFP?= =?us-ascii?q?0ABMgGBBQEBjyABAQ?= X-IronPort-AV: E=Sophos;i="5.64,490,1559545200"; d="scan'208";a="430754063" Received: from optima.cs.arizona.edu ([192.12.69.5]) by mails1n0out.email.arizona.edu with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2019 12:17:06 -0700 Received: from lectura.cs.arizona.edu (lectura.cs.arizona.edu [192.12.69.186]) by optima.cs.arizona.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id x8AJH6tt026822 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 10 Sep 2019 12:17:06 -0700 Received: from lectura.cs.arizona.edu (localhost [127.0.0.1]) by lectura.cs.arizona.edu (8.14.4/8.14.4/Debian-4.1ubuntu1.1) with ESMTP id x8AJH5Xk027878; Tue, 10 Sep 2019 12:17:05 -0700 Received: (from shijunxiao@localhost) by lectura.cs.arizona.edu (8.14.4/8.14.4/Submit) id x8AJH281027807; Tue, 10 Sep 2019 12:17:02 -0700 Message-Id: <201909101917.x8AJH281027807@lectura.cs.arizona.edu> From: Junxiao Shi Date: Tue, 10 Sep 2019 18:58:39 +0000 To: dev@dpdk.org Subject: [dpdk-dev] [RFC PATCH] kni: properly translate pa2va for cloned mbuf X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, KNI kernel module uses the difference between m->buf_addr and m->buf_iova to calculate userspace virtual address from physical address. This works for direct mbufs, but does not work for indirect (cloned) mbufs that come from another mempool. Transmitting a cloned mbuf may cause segmentation fault in userspace. Now, userspace KNI library writes the virtual address of each mbuf in m->userdata field, and KNI kernel module uses this field to restore virtual address before putting mbuf into free_q. This approach works for both direct and indirect mbufs. NOTE TO REVIEWER - DO NOT MERGE The idea of this change is at https://bugs.dpdk.org/show_bug.cgi?id=183#c4 Test case is at https://bugs.dpdk.org/show_bug.cgi?id=183#c5 I only modified kni_net_rx_normal function. If this approach is acceptable, I will modify kni_net_rx_lo_fifo, kni_net_rx_lo_fifo_skb, and kni_fifo_trans_pa2va(rx_q) as well. Bugzilla ID: 183 Signed-off-by: Junxiao Shi --- kernel/linux/kni/kni_net.c | 4 ++-- lib/librte_eal/linux/eal/include/rte_kni_common.h | 3 ++- lib/librte_kni/rte_kni.c | 8 ++++++-- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index 7bd3a9f1e..b34ed2ed4 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -341,7 +341,7 @@ kni_net_rx_normal(struct kni_dev *kni) kva = pa2kva(kni->pa[i]); len = kva->pkt_len; data_kva = kva2data_kva(kva); - kni->va[i] = pa2va(kni->pa[i], kva); + kni->va[i] = kva->va; skb = netdev_alloc_skb(dev, len); if (!skb) { @@ -367,7 +367,7 @@ kni_net_rx_normal(struct kni_dev *kni) kva = pa2kva(kva->next); data_kva = kva2data_kva(kva); /* Convert physical address to virtual address */ - prev_kva->next = pa2va(prev_kva->next, kva); + prev_kva->next = kva->va; } } diff --git a/lib/librte_eal/linux/eal/include/rte_kni_common.h b/lib/librte_eal/linux/eal/include/rte_kni_common.h index 37d9ee8f0..bc92c0067 100644 --- a/lib/librte_eal/linux/eal/include/rte_kni_common.h +++ b/lib/librte_eal/linux/eal/include/rte_kni_common.h @@ -84,7 +84,8 @@ struct rte_kni_mbuf { uint16_t data_len; /**< Amount of data in segment buffer. */ /* fields on second cache line */ - char pad3[8] __attribute__((__aligned__(RTE_CACHE_LINE_MIN_SIZE))); + void *va __attribute__((__aligned__(RTE_CACHE_LINE_MIN_SIZE))); + /**< Virtual address of this mbuf in userspace (overwrites userdata). */ void *pool; void *next; /**< Physical address of next mbuf in kernel. */ }; diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 4b51fb4fe..96a6f6af2 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -352,13 +352,15 @@ va2pa(struct rte_mbuf *m) static void * va2pa_all(struct rte_mbuf *mbuf) { - void *phy_mbuf = va2pa(mbuf); + void *phy_mbuf = (void*)rte_mempool_virt2iova(mbuf); struct rte_mbuf *next = mbuf->next; while (next) { - mbuf->next = va2pa(next); + mbuf->userdata = mbuf; + mbuf->next = (void*)rte_mempool_virt2iova(next); mbuf = next; next = mbuf->next; } + mbuf->userdata = mbuf; return phy_mbuf; } @@ -609,6 +611,8 @@ kni_allocate_mbufs(struct rte_kni *kni) offsetof(struct rte_kni_mbuf, buf_addr)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, next) != offsetof(struct rte_kni_mbuf, next)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, userdata) != + offsetof(struct rte_kni_mbuf, va)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != offsetof(struct rte_kni_mbuf, data_off)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != -- 2.17.1