From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by dpdk.org (Postfix) with ESMTP id 1F0812BD5 for ; Mon, 19 Nov 2018 18:04:23 +0100 (CET) Received: by mail-pl1-f193.google.com with SMTP id a14so10270311plm.12 for ; Mon, 19 Nov 2018 09:04:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Sa1moij+gbQnyQ0Hx68Y4khJ3OQX+ZMwIcOtV4xRbkM=; b=DuldFmys3WF3vFXccZLfV/CsnolcpGKIcGVe06NCZ6QAQ2QkJSH80oeydFN/T5hj/M fuS9jIjLlW4XQlouUSIykvwIZsYpomno7NeoWrcgNrbeF1RJTraivCmq8quVUtbSGeSn 0NYfzfXa0RqjIn7U7UPYIZvFaim0Z4fN3KeH/dFvRzzs2CPnoU/wdhfnRzFgHTD4q3IN Pu+hR31F8YbijZInYlS1L61GIXupoNtCjT0PDkNei604slxy29bQMJSbnqwbYCMYjneo Atp29mDXVjrfx2g3QlaBhjjygR5SDy2zwtfJqELTS8I76yJqxnpQlwhEcHbWeqqme1Vp ReVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Sa1moij+gbQnyQ0Hx68Y4khJ3OQX+ZMwIcOtV4xRbkM=; b=S21dJzNOgqSh9/j3cdbmZhC0HfXplbMvMG5sHAwNXES2+zzYXyYnGz7rxilGS6gPOb AzVUyy7JJZiP5N/37kp3TOcuttAGtMHTp2RjmAiydF1JeQdzj18LNzxuDuN6xLylLrvJ dbymAyrJQ5k+8mkVeCK2KepVLCYSsN48UseSS3tGTE4f0Cl+f33xeEUYo5vvm5Nz52Tp bZIDrOXU7tcvFm5GKkZtUlTd9YBADehFnj6sOlmzpNrIBIDmKDvYB5feE9MJJdv1+6ik yhE7wT7H8HVz0DO4w2aUygPJ0w7AGWodroyMCXHxTCTeymBq8aZqikHj2Atr7WIMkaJt DNlw== X-Gm-Message-State: AGRZ1gIgp/GzXRwue2aXqBAZP/HB6edgp8kYGZu8rK2GxCfmulVuYwUW DUhmWNk6gpLqIDvmvXcZ1Ar/AiyykuQ= X-Google-Smtp-Source: AJdET5czuIykKmZQX7TWwnULjXP3qKO9efAUO1NpD0cHqleG85wFrm/o1lzCGQTjwM58fn6efqllnw== X-Received: by 2002:a17:902:124:: with SMTP id 33-v6mr23316208plb.287.1542647063005; Mon, 19 Nov 2018 09:04:23 -0800 (PST) Received: from xeon-e3 (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id v89sm8638623pfk.12.2018.11.19.09.04.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 19 Nov 2018 09:04:22 -0800 (PST) Date: Mon, 19 Nov 2018 09:04:15 -0800 From: Stephen Hemminger To: "Burakov, Anatoly" Cc: Shahaf Shuler , "dev@dpdk.org" , Olga Shern , Yongseok Koh , "pawelx.wodkowski@intel.com" , "gowrishankar.m@linux.vnet.ibm.com" , "ferruh.yigit@intel.com" , Thomas Monjalon , "arybchenko@solarflare.com" , "shreyansh.jain@nxp.com" Message-ID: <20181119090415.56a16eca@xeon-e3> In-Reply-To: References: <6c7243cd-5370-846b-2999-5ae34722f640@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC] ethdev: introduce DMA memory mapping for external memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Nov 2018 17:04:24 -0000 On Thu, 15 Nov 2018 10:59:36 +0000 "Burakov, Anatoly" wrote: > > Unfortunately it cannot be done at least w/ Mellanox. > > In Mellanox the kernel driver is the one which maps the memory. The mapping returns a key which identify a memory region which was just registered to the device. > > There is a complete separation between the ports, meaning one port mapping cannot be used by in the other port, even if the key is known. > > > > The separation is not only in ports, but also in processes (two primary ones, for secondary we have a way to share). If two process work on the same device, the must register the memory independently. > > Ah, OK. > > So, we're right back to where we started. Right now, external memory > expects to behave the same way as all other memory - you don't need to > perform DMA mapping for it. > > That said, part of the reason *why* it was done that way was because > there is no way to trigger VFIO DMA mapping for NXP (or was it MLX?) > devices. If you look at initial versions of the patchset, the DMA > mapping was actually done manually. Then, i became convinced that doing > this automatically is the way to go, both because it erases the > usability differences as far as memory types are concerned, and because > it enables whatever services that are subscribing to memory events to > receive notifications about external memory as well (i.e. consistency). > > Given that it's still an experimental API, we can tinker with it all we > like, so it's not set in stone. However, i would really like to keep the > current automagic thing, because DMA mapping may not be the only user of > memory callbacks - they can be used for debug purposes, or for any other > things. If you look at the netvsc PMD, you will discover it to has the kernel setup the receive memory area. There is some flexibility about where it is mmap'd but the memory is coming from pinned kernel memory.