From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f51.google.com (mail-vk0-f51.google.com [209.85.213.51]) by dpdk.org (Postfix) with ESMTP id 4B2B56910 for ; Wed, 18 May 2016 12:28:04 +0200 (CEST) Received: by mail-vk0-f51.google.com with SMTP id s184so55126975vkb.3 for ; Wed, 18 May 2016 03:28:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=rsP7DwJvRczrpFsx0F+EwsOJgFP3dnt4s0D6xSoWnyM=; b=eBUVcA4gcQjzWrWuw4tlZS2avr/mVrzjSYqsCsKyTnpkOJNn2Gp2nyKeRb2i4Ouxj9 XGPqv6WtWTc0nr3cX+JDTnulRBrMdq3Lb2QsouljII8MbrKyL1hmk4ywBV1XbDnfwMVX jVaMreqYj0lwuhYWdnm592+3NWJQoLXQ267WAHizeBsFieg84JJkFH5v1zmH+YpG1M/g 6l/j4M04aBOIO31nI8i0razDvsvDqBDntvPnZKw0ARyANx70Hcy9smcc4UaA6SFyR6xU I+kxA8qO5+uFfsZfuo4Vyj6S1RLzkjcoMZMIHof1lNGGH/IZOfnX805GIRywJjMpkYQA MUEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=rsP7DwJvRczrpFsx0F+EwsOJgFP3dnt4s0D6xSoWnyM=; b=KP7fX5tF+yHdNoYTnfgwBRkWzQ8fa5GW0ku7949sCRzl8wtViGFon81eZUJlr+cHrI /1bQ/iJpjlXiIudD2o0DDfnCZo/h9ILJicQgFEfJ7KCrnXES6OTwfZTCxudvCwl/My1G 9MeObkRweFH8FrZ/kaRRpDgBcafYw03k13LGErXUx4hPC4qFUhuVdIG0sZS+LQGIxsTq k/CNEdZCMP6dvbB/rGDVw3MmXcn/cxhkNuo6BOFF7KvbqO/vzK5Qv1wv8+LOeJm4FJmF u4iptKCmLTU8N3Z1NsRdiWgEQDHpcCbuDyOSGR7t/uRtd09nD8+fZfVwSE2kF/V5n+J2 yBOQ== X-Gm-Message-State: AOPr4FXYi426+TEJOmCK2CM1y7pSCSOnksUETKvN53zSjwUO2n8evIzl06JkFk9gXzsB9sSaxGcIbkGQ+1NmE8wm MIME-Version: 1.0 X-Received: by 10.176.64.133 with SMTP id i5mr3636984uad.12.1463567283618; Wed, 18 May 2016 03:28:03 -0700 (PDT) Received: by 10.103.112.129 with HTTP; Wed, 18 May 2016 03:28:03 -0700 (PDT) In-Reply-To: <1500486.8lzTDt5Q91@xps13> References: <1500486.8lzTDt5Q91@xps13> Date: Wed, 18 May 2016 11:28:03 +0100 Message-ID: From: Alejandro Lucero To: Thomas Monjalon , "Burakov, Anatoly" Cc: sergio.gonzalez.monroy@intel.com, dev Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] memory allocation requirements X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 10:28:04 -0000 On Wed, Apr 13, 2016 at 5:03 PM, Thomas Monjalon wrote: > After looking at the patches for container support, it appears that > some changes are needed in the memory management: > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/32786/focus=32788 > > I think it is time to collect what are the needs and expectations of > the DPDK memory allocator. The goal is to satisfy every needs while > cleaning the API. > Here is a first try to start the discussion. > > The memory allocator has 2 classes of API in DPDK. > First the user/application allows or requires DPDK to take over some > memory resources of the system. The characteristics can be: > - numa node > - page size > - swappable or not > - contiguous (cannot be guaranteed) or not > - physical address (as root only) > Then the drivers or other libraries use the memory through > - rte_malloc > - rte_memzone > - rte_mempool > I think we can integrate the characteristics of the requested memory > in rte_malloc. Then rte_memzone would be only a named rte_malloc. > The rte_mempool still focus on collection of objects with cache. > > If a rework happens, maybe that the build options CONFIG_RTE_LIBRTE_IVSHMEM > and CONFIG_RTE_EAL_SINGLE_FILE_SEGMENTS can be removed. > The Xen support should also be better integrated. > > Currently, the first class of API is directly implemented as command line > parameters. Please let's think of C functions first. > The EAL parameters should simply wrap some API functions and let the > applications tune the memory initialization with a well documented API. > > Probably that I forget some needs, e.g. for the secondary processes. > Please comment. > Just to mention VFIO IOMMU mapping should be adjusted for just those memsegs physically contiguous which rte_pktmbuf_pool_create will allocate along with those hugepages backing driver/device descriptor rings. Mapping all the memsegs is not a performance issue but I think it is the right thing to do. Maybe some memseg flag like "DMA_CAPABLE" or similar should be used for IOMMU mapping. Other question is avoiding to use mbufs from no "DMA_CAPABLE" segments with a device. I'm thinking about an DPDK app using a virtio network driver and a device-backed PMD at the same time what could be a possibility for having best of both worlds (intra-host and inter-host VM communications).