From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0056.outbound.protection.outlook.com [104.47.36.56]) by dpdk.org (Postfix) with ESMTP id 3521A1B213 for ; Wed, 1 Nov 2017 07:54:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=fDqZQFuvlNx9cCAIdhXcuq3qsUyyXAutfXO2rqF8ih8=; b=RDBtLTqes1yjRIfvnf5MJuBkEohcInyV/AYZQnwq/lLm4pSxyKOFblGEcckpGpUNJg2/Dyzj2jTN2DfMb2GIMyN+/EvkD+qfwMiCV+lDEevawTb9xfW7NqaTxVBGGLezIaWnk5zh4GSNTRYBNW1kC4Y3bjMX42Uyf+MTj3pTPYA= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (106.201.40.78) by BN3PR07MB2515.namprd07.prod.outlook.com (10.167.4.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.197.13; Wed, 1 Nov 2017 06:54:52 +0000 Date: Wed, 1 Nov 2017 12:24:32 +0530 From: Jerin Jacob To: Ferruh Yigit Cc: Thomas Monjalon , Bruce Richardson , Sergio Gonzalez Monroy , dev@dpdk.org, Jianfeng Tan , Santosh Shukla Message-ID: <20171101065431.GA25152@jerin> References: <20171101010726.17781-1-ferruh.yigit@intel.com> <20171101040251.GA13569@jerin> <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> User-Agent: Mutt/1.9.1 (2017-09-22) X-Originating-IP: [106.201.40.78] X-ClientProxiedBy: MAXPR0101CA0017.INDPRD01.PROD.OUTLOOK.COM (10.174.62.27) To BN3PR07MB2515.namprd07.prod.outlook.com (10.167.4.140) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1d99206a-14fb-4e19-5fa3-08d520f57599 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(2017052603199); SRVR:BN3PR07MB2515; X-Microsoft-Exchange-Diagnostics: 1; BN3PR07MB2515; 3:T3HsyvRmZ92Ww2aS/Uxr1l708vbC3PBs9lORJlu/oJSu62ouGnPkiO2uc/dFcFjxfFroRgNDFli2xl6TBNttoCBZ1O07DFoyk3TEo/5H69z5XBG0zW3wLa5ILQWyDBtDC/kwvE0fmhOTO0uZaJgYMVGxULp9Q2bODx7YXBF12UAxECjb1332Z1Yk7w3jJgDC9SAWoWULiPeFhwEoepBPQlqXIkbkf3B09ZB4W88TB9QfbTzIrsfXucwe5J+uoCQ3; 25:0TgYrD286188Xd6X2hBff9f3WIMsIPFE+BgCxrcARrGNsNY6S5haXSJOzO6R2Ck1dHYLvDdgxCBFiseuW3SfjI8ACO/bVD5mkcC+IMvr5JSeMCLPP8nuL1CoAP4nsRjVYBDBHg7hzHHLnpq4H7vwP8fyvFEd9+XSNhu5D+N9Yvz+2QzpkJc/N83AkXmhsBifLZuEm0cbhj9sHlP+E7A16TD8pyl59oZ8+vomQYXQCMqBhkqEK7fdfbRPxM0RQ5Cb299i0qFmHgmDlTszSODTsFKRccaoW1yNfzuRJZOahyxTGyCI3aDNHyRF2XUUv5nCjCbROGmf7ZyIaIQUcnKV6g==; 31:YBtWyfsJ9SarscccDpmbjUz2kCp3YVG87NCCwjGWOIj4ER0Z+PUGVmpj9g941iPRQVGYye8xnZgQGXXDKFh+pPD7xR4w0YhXbCCAMafESLsM8mlkuj7jJVDQGk8JseHq9gJY7K6acXKuYtlbCm7eqxCdVN9zcb1nWpcr3afW3w9p277yiPNSEG/GWEE1IPGZ9HCuZJyXuu64Ld3lOMvj9qbvO1fX2IYL9yWCqt92Gag= X-MS-TrafficTypeDiagnostic: BN3PR07MB2515: X-Microsoft-Exchange-Diagnostics: 1; BN3PR07MB2515; 20:5JWyNv43oODeihRWFbENMI3OUjIgzYF2DXu9CmTycMqv2dzNFxXIT6DQ+N6ClteccKRgNdlSJkz5Rpq6Ssilu64qf2BrXsb/TgL9xfFjo3yJtPZlu0tfWl5AursRLVc9hAfH4nrfDewGgZxZtn5rBi7rWL6Bb0FcqWkcqzK33SLeePsFIirwbZ6JFooHsWnk2VvkLb9H+jMJZ24KsJqLiJl52/XOJlCge77+m5iKERs0ypx5Nwqo6udK59LF6MiqKU9sK40IF9XcrDmOJHLKWpj9xuOfA3yqMukSyONSwtOaA0qxgqTHkAn2jCuaoTmfzUe1bX/Z6aDhsbhPm2XAnlQ6dyhDXN9JbvnYFYNMYsBTHkVvHZivqu8XyIwZYNmRuFV5TlW/GId0BpmpXbt8e7fJFyTMbj63tI/YweWIEG9xLBntA2Ih/vZkuZ1UsCWwgpjtpI4/6O2C7uAls81hLvI8Rn2cBvt8jaJNIhQ8o8mP6ZtZTNPM2NuCLun0c7xsN3yspEcisvkBIwUb3SZoTDHFmFZM8rrhVRtyw7ESbao8+hNUNIWxYeb3I9OLGlo4A9xMmrUrNfYpHy5DoXQTtrMoV0qPJcfxKwCMUqIGM84= X-Exchange-Antispam-Report-Test: UriScan:(190756311086443)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(10201501046)(100000703101)(100105400095)(3231020)(3002001)(6041248)(20161123562025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN3PR07MB2515; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN3PR07MB2515; X-Microsoft-Exchange-Diagnostics: 1; BN3PR07MB2515; 4:BnrpcGqUQNuD8hqFi9z/GpB6sxI1FhYv1KLrJxmXhj7Ixg7jWMOhLdxg82KOxmMx4jfTS8w7fz4y+aEbkYZyYqCMXAuzbtxH+cwNOShz3Bi4+0IIm4jufxU7z1SzRYUk/UQ3sjjEYuH+21Dhk2IcPujTh2aJNbDGf5jZ3X4l8WGxXh+LppGxl+0vYMCuaZ1WAb+KATJDJdRqGL0r6CFTFJ/FCz4ZdS5WKp1IJgUnAQ2Bl6C65Ekd3fBxCph2h5AtLUFjYtrWbpYD/4dqPFzP4x1FcBQWPqkqgQbXFXIg4usk7MIB7bLjbvaDXE7CsUmJqBmNQwbKumgXfmVDrT/c4KAy9xsHpVCWSokGOCbHTxQ= X-Forefront-PRVS: 0478C23FE0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(199003)(24454002)(189002)(13464003)(2950100002)(42882006)(25786009)(4326008)(23726003)(6246003)(107886003)(53376002)(6496005)(8676002)(8936002)(81166006)(81156014)(2906002)(68736007)(3846002)(6116002)(53546010)(9686003)(16586007)(55016002)(54906003)(58126008)(1076002)(6306002)(97736004)(53936002)(316002)(1720100001)(33716001)(6916009)(478600001)(47776003)(66066001)(54356999)(76176999)(50986999)(5009440100003)(229853002)(101416001)(50466002)(33656002)(106356001)(966005)(105586002)(72206003)(83506002)(5660300001)(6666003)(189998001)(305945005)(7736002)(16526018)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR07MB2515; H:jerin; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR07MB2515; 23:qiyanrP5zHrQ4Z5eZ9KPsEQ+dqT10w4qA/kKQk2Zu?= =?us-ascii?Q?/2YGI/KbQXHScq6Vze1POiun/AgpO45/jOcGj4ahBgnuNnMXcqeLIa7NH4KR?= =?us-ascii?Q?6rmsJF647inJ9IynFk3azJVovx668iE1NWNOHn7+VNexKgqdu0zfLvVU2nTF?= =?us-ascii?Q?DRU70M9Eyv1tIiRswkJpJW85/pihziJbDPkPZCEeLZwKV3+UjpAcDgnenEa+?= =?us-ascii?Q?aP1UgVL17pGeM3PdLUZfXgV0za2WjHc9yk8adUCk5I/jrBrrxv9Sh1D3/AI+?= =?us-ascii?Q?sprdSjp72cX/CfZsUfNyYvf0pWb+7GIANWcsiaiIfyby4EtnAlid1wjRoQMG?= =?us-ascii?Q?y/hERo0KQ0zANXg02yyDPTG8ubUBkS6qIV8d2i3PAy+uiuOdCbot39phVk34?= =?us-ascii?Q?TYM6s96R07H4hvvorPmw5SGjhCFGxgLy6iRqASbDQJC+LY+n42GxBwZhvwuW?= =?us-ascii?Q?xCvhuXk6MLPTP6p7MF/+LxdBArvJTiI1s5oIz/UHO4Oxc9iQ8J46oqnydkzX?= =?us-ascii?Q?1CHh/00A7wVSp6ogHK8O3NYw2Kw+KW88CjkrA90deuD0539SkrwJHw2hJUG6?= =?us-ascii?Q?vKY+VsJ9387a3t6C4lDl6CUbpSwJjkx6LGFvgq6WH0T6zNTiwxUUTL4BGwpS?= =?us-ascii?Q?WET4sd6faQU9xogXHdd5o5M69fyXuGPC1TCFKAsZKOSTq8YldA37OGseVIkk?= =?us-ascii?Q?a5+hEN23X0Uv2xpoHxFu9OA64XW1as8/cZE2Y5o2uu7XOzcVEuUqZi6othOB?= =?us-ascii?Q?IfGaHC9l6VUvY99TwgKohTF35y+hPVcuf3mXuhj9JV8+bL7Yc7hH6BJhY/Pi?= =?us-ascii?Q?e9fLWOBpnW61yBENeJMBhF1hltZlooIlXboOwqy/GpVUXfNZpASKhWeEdwrU?= =?us-ascii?Q?nNSwtREzhc3WWU8BSyPq30GpcEFT1h0PAOM2FLMmQx11AZRCyIDeZRYVh74D?= =?us-ascii?Q?Y6AH4xUDuMIw3ZfRjltDsyYtCgrgxrSotERndjJEJowYVL6XjjGIjk6aIYOh?= =?us-ascii?Q?MpMtx62tZbg+ahTmfV8crh1bXDQ2J8EfeE1T50MM6MIgTgBecwOzB9LZwoX+?= =?us-ascii?Q?4ra9fjEXnoxsceJksTsrVbfw0GYnFWz5M1lGoIPx1k3KJRaxhCTGi6Iv+dD6?= =?us-ascii?Q?GppBDlSnVqoNRghstBL16XbTn2zeUS3GjtpXeo1gaA7KwPuABdGhRrZ6gE9u?= =?us-ascii?Q?3LZsCm34XWHTcQUYINRO4XYgOBb/W+GxAoKKWC8WaFDZxs250hdPkPfZ8gZC?= =?us-ascii?Q?hSckH18YTeSzwHbzVIPaz2AX9q/uD6oXfDclPMzbDcqaGxtopXPtEWSvSDui?= =?us-ascii?Q?q4jzTpg0WiRWwJwQ9sk13x3EFgSBpf/Sf95fYPz1o7qW2cSy4dteFXqmUCDW?= =?us-ascii?Q?+YTUO1wa9Kt8a6pnmruEoCBrqDKjsEV8RewA0f4eRfxv6fv?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR07MB2515; 6:GgkXJVgzfcBQCvrTKgRra1qkrkh42bpYZxyZRzoeM12IB8g2Y1oeXAxxmgDjFcuV8fxS2IfBVC6FHAxyQqOHVPPtxV15wfu1h6vxBeztJ2i3sFYdjCVEx0s/4s77lkgcaSLl/PVBEJOXpzlsLQ4CZNLx2+/dln7DhUl29ffxvy0/B8/FaGt25vPcYPo/un30JR9XQxLX/LHzi12y9c0WdX6Fe+CMkg6NsblEu94oDDra7Dt//yzWcmOeSbcv+T5vXMEHnupc1aKUpms0p8zB5pkGxxWny+RC0Yps0p2SLcEYw6qtfoqDukvA5NNmQs2fEHP55F/XaHsFHZOFjUefIImS+qVOx9J33nk3w0gapdc=; 5:Bc2Wl5kOsK2GBXWr4VKGho6lJs+1LHskgDV3VLpkUgJcnzL89N9dgSMZlPLMsum+KvTwzxoGda5Y1/7Gh4eXIh/c35HFhvTnQEbyCql37MqICKit9VjAgHEv+SXWLTakjX6BQDek5cYNL34kfFARyisvxUEMzKPvKwTFzi5fLOI=; 24:eJM3IGB78TvKB5U1QvjlmFGtD8zE/j5PZ7EQbiyhdVPCI+/iuzP5ctxV29XXUuWpheD2CP1obyDiNMUA/PnEKnqMfIRGZr9tQTa6VnpFyhk=; 7:BPXwDSy6HecWx77CWnnCT8ciafm5oDu8CPMa7YmpGAS65kUFQueFMuLeieQBelc+zHPs2VLC+1LnA7/hRgvQSIQvMyJc25zI9r0N69wv7Eox8wHM1Hq8MA61syJMFo2r2aPjpG9q4MiM7pl1A17r0o2ddYdFvNdUtod2PKEpHXmpc06RmXllNBmX0PNtuMbfPaA8ElPpeErw8xvLu8Zs/MKaRf9Dizem/ceqOxeHVJxnsPQPqizPHDHvUzvqxpEq SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2017 06:54:52.7355 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1d99206a-14fb-4e19-5fa3-08d520f57599 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR07MB2515 Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Nov 2017 06:54:58 -0000 -----Original Message----- > Date: Tue, 31 Oct 2017 23:21:18 -0700 > From: Ferruh Yigit > To: Jerin Jacob > CC: Thomas Monjalon , Bruce Richardson > , Sergio Gonzalez Monroy > , dev@dpdk.org, Jianfeng Tan > , Santosh Shukla > > Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 > Thunderbird/52.4.0 > > On 10/31/2017 9:02 PM, Jerin Jacob wrote: > > -----Original Message----- > >> Date: Wed, 1 Nov 2017 01:07:26 +0000 > >> From: Ferruh Yigit > >> To: Thomas Monjalon , Bruce Richardson > >> , Sergio Gonzalez Monroy > >> > >> CC: dev@dpdk.org, Ferruh Yigit , Jianfeng Tan > >> , Santosh Shukla > >> > >> Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > >> X-Mailer: git-send-email 2.13.6 > >> > >> Fix kernel crash with KNI because KNI requires physical addresses. > > > > The actual fix would be to make KNI IOMMU aware based on the DPDK mode. > > > > ie. On slow path, > > > > /* Get iommu domain for iova to physical addr conversion */ > > if (rte_eal_iova_mode() == RTE_IOVA_VA) > > kni->iommu_domain = iommu_get_domain_for_dev(dev); > > else > > kni->iommu_domain = NULL; > > > > On fast path, > > > > static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr) > > { > > /* Translation is installed only when IOMMU is present */ > > if (kni->iommu_domain) > > return iommu_iova_to_phys(kni->iommu_domain, dma_addr); > > return dma_addr; > > } > > > >> > >> A config option introduced to disable IOVA mode detection and to set it > >> to physical address by default. Disabling config option will enable IOVA > >> mode detection. > >> > >> When there is no intension to use KNI, it is safe to enable detection. > >> > >> Config option disable IOVA mode detection by default to be sure only who > >> is aware of result enable it. > >> > >> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy") > >> > >> Signed-off-by: Ferruh Yigit > >> --- > >> Cc: Jianfeng Tan > >> Cc: Santosh Shukla > >> Cc: Thomas Monjalon > >> --- > >> config/common_base | 5 +++++ > >> lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ > >> lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ > >> 3 files changed, 13 insertions(+) > >> > >> diff --git a/config/common_base b/config/common_base > >> index 82ee75456..903e7685b 100644 > >> --- a/config/common_base > >> +++ b/config/common_base > >> @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n > >> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n > >> > >> # > >> +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution > >> +# > >> +CONFIG_RTE_EAL_USE_PHYS_IOVA=y > > > > Defeat the purpose of all dynamic probing scheme. > > Either we can fix the KNI or revert the following patch for this release. > > > > http://dpdk.org/commit/f37dfab2 > > This commit just enables IOVA VA mode for Intel drivers, that is how I can able > to observe the issue, but it is not the source of the problem. Reverting that > commit will not solve KNI crash with any other PMD that enables IOVA VA mode. I don't understand why a PMD needs to enable IOVA_VA if it can support IOVA_PA. IMO, IOVA_VA should be enabled only for those device it can WORK ONLY on IOVA_VA mode. Forget about KNI, If we set CONFIG_RTE_EAL_USE_PHYS_IOVA as y then the normal stuff wont work for if PMD can operate only in IOVA_VA mode(like octeontx). Regarding the KNI crash, it can be avoid by first checking the exiting mode(rte_eal_iova_mode()). i.e since legacy driver like KNI need real physical address to work "now", it can grace full exit on the init time if mode == IOVA_VA; > > Related to the KNI, iommu is not involved at all, I am not clear with your above > suggestion, physical address is required for kernelspace - userspace communication. vhost-kernel addressed this case with IOVA as VA. I need to spend cycles on what takes to remove physical address dependency from KNI. But someone can add that support if it is required in future. > KNI uses physical address for two things: > > 1- Creates a buffer in userspace, a memzone, and shares its physical address to > the kernel, so that both kernel and app can access same buffer. > > The question is, even all devices supports IOVA VA mode, why creating a memzone > that has real physical address info is not possible anymore? > For KNI case this is not related at all with what devices supports. > > 2- For each mbuf that will be sent to kernel, dpdk app puts physical address of > that mbuf into shared buffer, so that kernel can access it. So that kernel can > access to mbuf data that can be coming from any mempool. > For this the physical address of the mbuf is required. > > > Overall KNI needs to know physical address of mbufs and memzone. > > This patch provides a way to have old behavior with a config option, so that KNI > can work and anyone who needs to create memzone with physical address can a way > to have it, and all PMDs will work fine. > > If there is no need for these, it is possible to disable config and dynamic > probing scheme is already there. > > Thanks, > ferruh