From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <thomas@monjalon.net>
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com
 [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id 9AB512935
 for <dev@dpdk.org>; Tue,  4 Jul 2017 12:10:47 +0200 (CEST)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailout.nyi.internal (Postfix) with ESMTP id 2287C20A52;
 Tue,  4 Jul 2017 06:10:46 -0400 (EDT)
Received: from frontend1 ([10.202.2.160])
 by compute1.internal (MEProxy); Tue, 04 Jul 2017 06:10:46 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h=
 cc:content-transfer-encoding:content-type:date:from:in-reply-to
 :message-id:mime-version:references:subject:to:x-me-sender
 :x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=DvX2rrcRM662UHD
 ACGepIcMo/5WdjqiKsuzC5hwLFAI=; b=iHhoiuTX9KtH0H87P+Y5FVgE8XxMuM1
 YdacuY8r33yYxSGFKjUfOJmIDUiA+XcXNGWob1EpTsFcUal1qWG18znTCqdIS9p/
 dsAPS00t/U7EsYZz+FeFSpoUSPLAgMCeXc4RSf0Jrzx3fghFX0pUVUhmivoapbSk
 NK81F33QGoG0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-transfer-encoding:content-type
 :date:from:in-reply-to:message-id:mime-version:references
 :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=
 fm1; bh=DvX2rrcRM662UHDACGepIcMo/5WdjqiKsuzC5hwLFAI=; b=HVWeD2Rc
 Biztkkok/fx/ZMWsM/O34DdWeNjUrH6bCyG5mh1q5+7wU0xv3rmv11ugtC7zgJct
 rK0qlPyZ8J4xD96kqgyc3bqfRE//uaugHAd+a5XN7ySRksUAek4s6F1pg6UIsXTp
 W579ZvVArpBg4K+hcEVCMGg0yw4DcLziTHyEf03tO+rzHndWf+mGsxEFBfhEWEFI
 YX0Akas30b67nIFr8UcsuURrK7vicFQHP3z+kxeAhU4JSsHAqL7OaZ6h5I/VJJMS
 Ig3Yq1jmedh5vJi1jdh0eXCVaeVK4txF4/jmX5IQYL3uqRYXTy13EnWAg3jdQNr0
 M0836YcAnf+BVw==
X-ME-Sender: <xms:pmlbWSamr21G6aGJxVjSah75Itvh3-ZgXSwl4Iz27_1djXcvsmL4Ig>
X-Sasl-enc: JHejR/ItKVV6LXu4ALW7t458X6C6f96uBalOZMEzNVxz 1499163045
Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184])
 by mail.messagingengine.com (Postfix) with ESMTPA id C0AE67E816;
 Tue,  4 Jul 2017 06:10:45 -0400 (EDT)
From: Thomas Monjalon <thomas@monjalon.net>
To: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Cc: dev@dpdk.org, bruce.richardson@intel.com, jerin.jacob@caviumnetworks.com,
 hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com,
 jblunck@infradead.org, olivier.matz@6wind.com, jianbo.liu@linaro.org,
 anatoly.burakov@intel.com, sergio.gonzalez.monroy@intel.com
Date: Tue, 04 Jul 2017 12:10:44 +0200
Message-ID: <1695601.K0fVzJ8uRQ@xps>
In-Reply-To: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com>
References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
Subject: Re: [dpdk-dev] [PATCH 00/10] Infrastructure to detect iova mapping
	on the bus
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jul 2017 10:10:47 -0000

Hi Santosh,
Let's try to make this proposal clearer in order to have some reviews.

08/06/2017 13:05, Santosh Shukla:
> Q) Why do we need such infrastructure?
> 
> A) Some NPU hardware like OCTEONTX follows push model to get the packet
> from the pktio device. Where packet allocation and freeing done
> by the HW. Since HW can operate only on IOVA with help of SMMU/IOMMU,

Some readers may not know IOVA: IO Virtual address.
Some explanations:
	https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt
	http://vfio.blogspot.fr/2014/08/iommu-groups-inside-and-out.html

It must be said that SMMU is equivalent to IOMMU for ARM:
	https://developer.arm.com/products/system-ip/system-controllers/system-memory-management-unit

> when packet receives from the Ethernet device, it is the IOVA address
> (which is PA in existing scheme).

You mean that we are currently using only Physical Address (PA)?

> Mapping IOVA as PA is expensive on those HW, where every packet
> needs to be converted to VA from PA/IOVA.

Please, could you explain how and where addresses are converted currently?

> This patchset proposes the method to autodetect the preferred
> IOVA mode for a device. Summary of IOVA scheme:
> - If all the devices are iommu capable and support IOMMU
>   capable driver then selects IOVA_VA.
> - If any of the devices are non-iommu then use default IOVA
>   scheme ie. IOVA_PA.
> - If no device found then IOVA scheme would be
>   IOVA_DC (Don't care).

I think you should better describe these modes and how they behave.

> To achieve that, two global APIs introduced:
> - rte_bus_get_iommu_class
> - rte_pci_get_iommu_class
> 
> Return values for those APIs are:
> enum rte_iova_mod {
>         RTE_IOVA_DC, /* Don't care */
>         RTE_IOVA_PA,
>         RTE_IOVA_VA
> }
> 
> Those are the bus policy for selecting IOVA mode. In case user
> want to override bus IOVA mapping then added an EAL option
> "--iova-mode=<string>". User to pass string format 'pa' --> IOVA_PA,
> 'va' --> IOVA_VA.
> 
> To support new eal option, adding global API:
> - rte_eal_iova_mode
> 
> Patch Summary:
> 2) 1st - 2th patch: Adds infrastructure in linuxapp and bsdapp
> layer.
> 1) 3rd patch: Introduces global bus api named rte_bus_get_iommu_class.
> 3) 4th patch: Add new eal option called --iova-mode=<mode-string>.
> 4) 5th - 6th patch: Logic to detect iova scheme.
> 5) 9th patch: Check IOVA mode before programing vfio dma_map.iova.
> Default scheme is IOVA_PA.
> 6) 10th-12th patch: Check for IOVA_VA mode in below APIs
>         - rte_mem_virt2phy
>         - rte_mempool_virt2phy
>         - rte_malloc_virt2phy
> If set then return paddr=vaddr, else return value from default
> implementation.