From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id C496E19F5 for ; Mon, 17 Sep 2018 12:07:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Sep 2018 03:07:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,385,1531810800"; d="scan'208";a="84154100" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.55]) ([10.237.220.55]) by orsmga003.jf.intel.com with ESMTP; 17 Sep 2018 03:07:18 -0700 To: Shahaf Shuler , "dev@dpdk.org" Cc: "laszlo.madarassy@ericsson.com" , "laszlo.vadkerti@ericsson.com" , "andras.kovacs@ericsson.com" , "winnie.tian@ericsson.com" , "daniel.andrasi@ericsson.com" , "janos.kobor@ericsson.com" , "srinath.mannam@broadcom.com" , "scott.branden@broadcom.com" , "ajit.khaparde@broadcom.com" , "keith.wiles@intel.com" , "bruce.richardson@intel.com" , Thomas Monjalon References: From: "Burakov, Anatoly" Message-ID: <9c763791-2c10-fd36-7f50-3df5efbc20ab@intel.com> Date: Mon, 17 Sep 2018 11:07:17 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 00/16] Support externally allocated memory in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 10:07:31 -0000 On 13-Sep-18 8:44 AM, Shahaf Shuler wrote: > Hi Anatoly, > > First thanks for the patchset, it is a great enhancement. > > See question below. > > Tuesday, September 4, 2018 4:12 PM, Anatoly Burakov: >> Subject: [dpdk-dev] [PATCH 00/16] Support externally allocated memory in >> DPDK >> >> This is a proposal to enable using externally allocated memory in DPDK. >> >> In a nutshell, here is what is being done here: >> >> - Index internal malloc heaps by NUMA node index, rather than NUMA >> node itself (external heaps will have ID's in order of creation) >> - Add identifier string to malloc heap, to uniquely identify it >> - Each new heap will receive a unique socket ID that will be used by >> allocator to decide from which heap (internal or external) to >> allocate requested amount of memory >> - Allow creating named heaps and add/remove memory to/from those >> heaps >> - Allocate memseg lists at runtime, to keep track of IOVA addresses >> of externally allocated memory >> - If IOVA addresses aren't provided, use RTE_BAD_IOVA >> - Allow malloc and memzones to allocate from external heaps >> - Allow other data structures to allocate from externall heaps >> >> The responsibility to ensure memory is accessible before using it is on the >> shoulders of the user - there is no checking done with regards to validity of >> the memory (nor could there be...). > > That makes sense. However who should be in-charge of mapping this memory for dma access? > The user or internally be the PMD when encounter the first packet or while traversing the existing mempools? > Hi Shahaf, There are two ways this can be solved. The first way is to perform VFIO mapping automatically on adding/attaching memory. The second is to force user to do it manually. For now, the latter is chosen because user knows best if they intend to do DMA on that memory, but i'm open to suggestions. There is an issue with some devices and buses (i.e. bus/fslmc) bypassing EAL VFIO infrastructure and performing their own VFIO/DMA mapping magic, but solving that problem is outside the scope of this patchset. Those devices/buses should fix themselves :) When not using VFIO, it's out of our hands anyway. -- Thanks, Anatoly