From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 2104A5B12 for ; Tue, 20 Mar 2018 16:25:15 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Mar 2018 08:25:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,336,1517904000"; d="scan'208";a="35371791" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.63]) ([10.237.221.63]) by FMSMGA003.fm.intel.com with ESMTP; 20 Mar 2018 08:25:13 -0700 To: cys Cc: dev References: <528c4b67.fc.1620aaef223.Coremail.chaoys155@163.com> <60b8909c-b28d-eec4-110e-4c02c32cb087@intel.com> <428fa651.2c.16221ef2eaa.Coremail.chaoys155@163.com> From: Ferruh Yigit Message-ID: <230e41e3-c397-4088-fc47-3ecceb1468e5@intel.com> Date: Tue, 20 Mar 2018 15:25:13 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <428fa651.2c.16221ef2eaa.Coremail.chaoys155@163.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] kni: continuous memory restriction ? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2018 15:25:16 -0000 > > 在2018年03月13 22时57分, "Ferruh Yigit"写道: > > > On 3/9/2018 12:14 PM, cys wrote: > > Commit 8451269e6d7ba7501723fe2efd0 said "remove continuous memory > restriction"; > > > http://dpdk.org/browse/dpdk/commit/lib/librte_eal/linuxapp/kni/kni_net.c?id=8451269e6d7ba7501723fe2efd05745010295bac > > For chained mbufs(nb_segs > 1), function va2pa use the offset of previous mbuf > > to calculate physical address of next mbuf. > > So anywhere guarante that all mbufs have the same offset (buf_addr - > buf_physaddr) ? > > Or have I misunderstood chained mbufs? > > Hi, > > Your description is correct, KNI chained mbufs is broken if chained mbufs are > from different mempools. > > Two commits seems involved, in time order: > [1] d89a58dfe90b ("kni: support chained mbufs") > [2] 8451269e6d7b ("kni: remove continuous memory restriction") > > With current implementation, kernel needs to know physical address of the mbuf > to be able to access it. > For chained mbufs, first mbuf is OK but for rest kernel side gets the virtual > address of the mbuf and this only works if all chained mbufs are from same > mempool. > > I don't have any good solution indeed, but it is possible to: > a) If you are using chained mbufs, keep old limitation of using singe mempool > b) Serialize chained mbufs for KNI in userspace > On 3/14/2018 12:35 AM, cys wrote: > Thanks for your reply. > With your solution a), I guess 'single mempool' mean a mempool fit in one memseg > (continuous memory). Yes I mean physically continuous memory, a mempool from single memseg, otherwise it has same problem. > What about a mempool across many memsegs ? I'm afraid it's still not safe. > Just like this one: > -------------- MEMPOOL ---------------- > mempool @0x7ff9e4833d00 > flags=10 > pool=0x7ff9fbfffe00 > phys_addr=0xc4fc33d00 > nb_mem_chunks=91 > size=524288 > populated_size=524288 > header_size=64 > elt_size=2432 > trailer_size=0 > total_obj_size=2496 > private_data_size=64 > avg bytes/object=2496.233643 > > Zone 0: name:, phys:0xc4fdb7f40, len:0x34000, > virt:0x7ff9e49b7f40, socket_id:0, flags:0 > Zone 1: name:, phys:0xc4fc33d00, len:0x182100, > virt:0x7ff9e4833d00, socket_id:0, flags:0 > Zone 2: name:, phys:0xb22000080, len:0x16ffff40, > virt:0x7ffa3a800080, socket_id:0, flags:0 > Zone 3: name:, phys:0xc199ffe00, len:0x800180, > virt:0x7ff9fbfffe00, socket_id:0, flags:0 > Zone 4: name:, phys:0xc29c00080, len:0x77fff40, > virt:0x7ff9e5800080, socket_id:0, flags:0 > Zone 5: name:, phys:0xc22c00080, len:0x67fff40, > virt:0x7ff9ed200080, socket_id:0, flags:0 > Zone 6: name:, phys:0xc1dc00080, len:0x3bfff40, > virt:0x7ff9f4800080, socket_id:0, flags:0 > Zone 7: name:, phys:0xc1bc00080, len:0x1bfff40, > virt:0x7ff9f8600080, socket_id:0, flags:0 > Zone 8: name:, phys:0xbf4600080, len:0xffff40, > virt:0x7ffa1ea00080, socket_id:0, flags:0 > Zone 9: name:, phys:0xc0e000080, len:0xdfff40, > virt:0x7ffa06400080, socket_id:0, flags:0 > Zone 10: name:, phys:0xbe0600080, len:0xdfff40, > virt:0x7ffa32000080, socket_id:0, flags:0 > Zone 11: name:, phys:0xc18000080, len:0xbfff40, > virt:0x7ff9fd000080, socket_id:0, flags:0 > Zone 12: name:, phys:0x65000080, len:0xbfff40, > virt:0x7ffa54e00080, socket_id:0, flags:0 > Zone 13: name:, phys:0xc12a00080, len:0x7fff40, > virt:0x7ffa02200080, socket_id:0, flags:0 > Zone 14: name:, phys:0xc0d600080, len:0x7fff40, > virt:0x7ffa07400080, socket_id:0, flags:0 > Zone 15: name:, phys:0xc06600080, len:0x7fff40, > virt:0x7ffa0de00080, socket_id:0, flags:0 > ...