From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 0F47F2B93 for ; Wed, 6 Mar 2019 18:31:23 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2019 09:31:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,448,1544515200"; d="scan'208";a="149179719" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.252.14.113]) ([10.252.14.113]) by fmsmga002.fm.intel.com with ESMTP; 06 Mar 2019 09:31:21 -0800 To: Yangchao Zhou , dev@dpdk.org References: <20190228073010.49716-1-zhouyates@gmail.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJVBBMBAgA/AhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgBYhBNI2U4dCLsKE45mBx/kz60PfE2EfBQJbughWBQkHwjOGAAoJEPkz60Pf E2Eft84QAIbKWqhgqRfoiw/BbXbA1+qm2o4UgkCRQ0yJgt9QsnbpOmPKydHH0ixCliNz1J8e mRXCkMini1bTpnzp7spOjQGLeAFkNFz6BMq8YF2mVWbGEDE9WgnAxZdi0eLY7ZQnHbE6AxKL SXmpe9INb6z3ztseFt7mqje/W/6DWYIMnH3Yz9KzxujFWDcq8UCAvPkxVQXLTMpauhFgYeEx Nub5HbvhxTfUkapLwRQsSd/HbywzqZ3s/bbYMjj5JO3tgMiM9g9HOjv1G2f1dQjHi5YQiTZl 1eIIqQ3pTic6ROaiZqNmQFXPsoOOFfXF8nN2zg8kl/sSdoXWHhama5hbwwtl1vdaygQYlmdK H2ueiFh/UvT3WG3waNv2eZiEbHV8Rk52Xyn2w1G90lV0fYC6Ket1Xjoch7kjwbx793Kz/RfQ rmBY8/S4DTGn3oq3dMdQY+b6+7VMUeLMMh2CXYO9ErkOq+qNTD1IY+cBAkXnaDbQfz0zbste ZGWH74FAZ9nCpDOqbRTrBL42aMGhfOWEyeA1x7+hl6JZfabBWAuf4nnCXuorKHzBXTrf7u7p fXsKQClWRW77PF1VmzrtKNVSytQAmlCWApQIw20AarFipXmVdIjHmJPU611WoyxZPb4JTOxx 5cv9B+nr/RIB+v5dcStyHCCwO1be7nBDdCgd4F6kTQPLuQINBFfWTL4BEACnNA29e8TarUsB L5n6eLZHXcFvVwNLVlirWOClHXf44o2KnN3ww+eBEmKVfEFo9MSuGDNHS8Zw1NiGMYxLIUgd U6gGrVVs/VrQWL82pbMk6jCj98N+BXIri+6K1z+AImz7ax7iF1kDgRAnFWU0znWWBgM2mM8Y gDjcxfXk4sCKnvf6Gjo08Ey5zmqx7dekAKU2EEp8Q1EJY3jbymLdZWRP4AFFMTS1rGMk0/tt v71NBg1GobCcbNfn9chK/jhqxYhAJqq86RdJQkt3/9x1U1Oq0vXCt4JVVHmkxePtUiuWTTt+ aYlUAsKYZsWvncExvw77x2ArYDmaK0yfjh37wp0lY7DOJHFxoyT8tyWZlLci/VMRG2Ja33xj 0CN4C1yBg+QDeV3QFxQo42iA/ykdXPUR3ezmsND3XKvVLTC4DNb3V/EZQ7jBj64+bEK0VW4G B31VP00ApNQvSoczsIOAKdk97RNbpmPw6q10ILIB+9T1xbnFYzshzGF17oC0/GENIHATx8vZ masOZoDiOZQpeneLgnFE9JfzhLTxv6wNZcc/HLXRQVTkDsQr8ERtkAoHCf1E5+b5Yr7pfnE4 YuhET746o25S53ELUYPIs49qoJsEJL34/oexMfPGyPIlrbufiNyty5jc/1MRwUlhJlJ5IOHy ZUa+6CLR7GdImusFkPJUJwARAQABiQI8BBgBAgAmAhsMFiEE0jZTh0IuwoTjmYHH+TPrQ98T YR8FAlu6CHAFCQXE7zIACgkQ+TPrQ98TYR9nXxAAqNBgkYNyGuWUuy0GwDQCbu3iiMyH1+D7 llafPcK4NYy1Z4AYuVwC9nmLaoj+ozdqS3ncRo57ncRsKEJC46nDJJZYZ5LSJVn63Y3NBF86 lxQAgjj2oyZEwaLKtKbAFsXL43jv1pUGgSvWwYtDwHITXXFQto9rZEuUDRFSx4sg9OR+Q6/6 LY+nQQ3OdHlBkflzYMPcWgDcvcTAO6yasLEUf7UcYoSWTyMYjLB4QuNlXzTswzGVMssJF/vo V8lD1eqqaSUWG3STF6GVLQOr1NLvN5+kUBiEStHFxBpgSCvYY9sNV8FS6N24CAWMBl+10W+D 2h1yiiP5dOdPcBDYKsgqDD91/sP0WdyMJkwdQJtD49f9f+lYloxHnSAxMleOpyscg1pldw+i mPaUY1bmIknLhhkqfMmjywQOXpac5LRMibAAYkcB8v7y3kwELnt8mhqqZy6LUsqcWygNbH/W K3GGt5tRpeIXeJ25x8gg5EBQ0Jnvp/IbBYQfPLtXH0Myq2QuAhk/1q2yEIbVjS+7iowEZNyE 56K63WBJxsJPB2mvmLgn98GqB4G6GufP1ndS0XDti/2K0o8rep9xoY/JDGi0n0L0tk9BHyoP Y7kaEpu7UyY3nVdRLe5H1/MnFG8hdJ97WqnPS0buYZlrbTV0nRFL/NI2VABl18vEEXvNQiO+ vM8= Message-ID: <9c7dcfd6-d55c-f5f6-b82c-461bc773dee4@intel.com> Date: Wed, 6 Mar 2019 17:31:21 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.2 MIME-Version: 1.0 In-Reply-To: <20190228073010.49716-1-zhouyates@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH] kni: fix possible kernel crash with va2pa X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2019 17:31:24 -0000 On 2/28/2019 7:30 AM, Yangchao Zhou wrote: > va2pa depends on the physical address and virtual address offset of > current mbuf. It may get the wrong physical address of next mbuf which > allocated in another hugepage segment. Hi Yangchao, The problem you described seems valid, when current mbuf and the mbuf pointed bu next pointer from different (huge)pages, address calculation will be wrong. Can you able to reproduce the issue, or recognized the problem theoretically? > > Signed-off-by: Yangchao Zhou > --- > kernel/linux/kni/kni_net.c | 16 ++-------------- > .../eal/include/exec-env/rte_kni_common.h | 4 ++++ > lib/librte_kni/rte_kni.c | 15 ++++++++++++++- > 3 files changed, 20 insertions(+), 15 deletions(-) > > diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c > index 7371b6d58..caef8754f 100644 > --- a/kernel/linux/kni/kni_net.c > +++ b/kernel/linux/kni/kni_net.c > @@ -61,18 +61,6 @@ kva2data_kva(struct rte_kni_mbuf *m) > return phys_to_virt(m->buf_physaddr + m->data_off); > } > > -/* virtual address to physical address */ > -static void * > -va2pa(void *va, struct rte_kni_mbuf *m) > -{ > - void *pa; > - > - pa = (void *)((unsigned long)va - > - ((unsigned long)m->buf_addr - > - (unsigned long)m->buf_physaddr)); > - return pa; > -} > - > /* > * It can be called to process the request. > */ > @@ -363,7 +351,7 @@ kni_net_rx_normal(struct kni_dev *kni) > if (!kva->next) > break; > > - kva = pa2kva(va2pa(kva->next, kva)); > + kva = pa2kva(kva->next_pa); > data_kva = kva2data_kva(kva); > } > } > @@ -545,7 +533,7 @@ kni_net_rx_lo_fifo_skb(struct kni_dev *kni) > if (!kva->next) > break; > > - kva = pa2kva(va2pa(kva->next, kva)); > + kva = pa2kva(kva->next_pa); > data_kva = kva2data_kva(kva); > } > } > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > index 5afa08713..608f5c13f 100644 > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > @@ -87,6 +87,10 @@ struct rte_kni_mbuf { > char pad3[8] __attribute__((__aligned__(RTE_CACHE_LINE_MIN_SIZE))); > void *pool; > void *next; > + union { > + uint64_t tx_offload; > + void *next_pa; /**< Physical address of next mbuf. */ > + }; This will cause overwrite the 'tx_offload' via 'next_pa', we don't use tx_offload in KNI but not sure about removing potential use for future. What do you think about converting 'm->next' to physical address before putting them into 'rx_q', and in kernel side after data copied to skb convert 'm->next' back to virtual address before putting it into 'free_q' ? I think both address conversion can be possible to do, a little tricky because address conversion should be calculated in next mbuf and previous mbuf->next in the chain should be updated. > }; > > /* > diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c > index 73aeccccf..1aaebcfa1 100644 > --- a/lib/librte_kni/rte_kni.c > +++ b/lib/librte_kni/rte_kni.c > @@ -353,6 +353,17 @@ va2pa(struct rte_mbuf *m) > (unsigned long)m->buf_iova)); > } > > +static void * > +va2pa_all(struct rte_mbuf *m) > +{ > + struct rte_kni_mbuf *mbuf = (struct rte_kni_mbuf *)m; > + while (mbuf->next) { > + mbuf->next_pa = va2pa(mbuf->next); > + mbuf = mbuf->next; > + } > + return va2pa(m); > +} > + > static void > obj_free(struct rte_mempool *mp __rte_unused, void *opaque, void *obj, > unsigned obj_idx __rte_unused) > @@ -550,7 +561,7 @@ rte_kni_tx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned num) > unsigned int i; > > for (i = 0; i < num; i++) > - phy_mbufs[i] = va2pa(mbufs[i]); > + phy_mbufs[i] = va2pa_all(mbufs[i]); > > ret = kni_fifo_put(kni->rx_q, phy_mbufs, num); > > @@ -607,6 +618,8 @@ kni_allocate_mbufs(struct rte_kni *kni) > offsetof(struct rte_kni_mbuf, pkt_len)); > RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != > offsetof(struct rte_kni_mbuf, ol_flags)); > + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != > + offsetof(struct rte_kni_mbuf, tx_offload)); > > /* Check if pktmbuf pool has been configured */ > if (kni->pktmbuf_pool == NULL) { >