From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0041.outbound.protection.outlook.com [104.47.41.41]) by dpdk.org (Postfix) with ESMTP id 1B5433772 for ; Thu, 6 Jul 2017 13:19:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=nAE7+rogu8hkn44jSM0ujECL0DRnCFRQ0WhFKknxGeY=; b=Kc49e4ugV+jeRbXYntMwv6wEQB7Jcwh9wcH8zMDi97QcgTBwCEN2Bwfyp3mwKQ2HU7Lg8Pmxj398rsFL56k5lTI7DEtkzNbW1DMeBsaXgAQfAyE7J7fAsAqHCRD4tEMHyYcIGB2emVtV4h960pmbPBvJJmUKwJM/ZBnFFRfuiLA= Authentication-Results: 6wind.com; dkim=none (message not signed) header.d=none;6wind.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from [10.160.0.43] (111.93.218.67) by CY4PR07MB3095.namprd07.prod.outlook.com (10.172.115.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1220.11; Thu, 6 Jul 2017 11:19:20 +0000 To: Maxime Coquelin , Jerin Jacob References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> <20170608110513.22548-8-santosh.shukla@caviumnetworks.com> <730e333b-a9ab-df8b-cf7a-1e0186c6152d@redhat.com> <20170705154314.GA4635@jerin> <2fe366fb-15fa-f754-458e-3f4e8be18699@redhat.com> <20170706094939.GA1709@jerin> <89425d75-3f79-d3e8-f0b1-330292866bbb@redhat.com> Cc: thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com From: santosh Message-ID: Date: Thu, 6 Jul 2017 16:49:06 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <89425d75-3f79-d3e8-f0b1-330292866bbb@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MA1PR01CA0078.INDPRD01.PROD.OUTLOOK.COM (10.174.56.18) To CY4PR07MB3095.namprd07.prod.outlook.com (10.172.115.9) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fd3a0a50-1e00-427c-4773-08d4c460db1f X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CY4PR07MB3095; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 3:b2wSBSrzDY1tKTa7PRszaV3go4WErVmqf/BjWpYr7ixRf4qtHqFpPH0fU+yXlRv9Dxu19M2akaSDV+Ip472E37nBMHQYcodoRDQy6DmaqdnQMHa8nxVLID+1O9EN4bpjgfPG6B8su89onjrnALZM3dmQrBDW//jkUd9yERqLwpqsHZzlWqNKbZRrQeDKTj90/jyl7uujERfRwwbs0hS4z8I5Y+r0TtBZQwqAkKw+dnvKB+Jdai2jH6ilycc/Ptc+y9JlOHl5chhK0lifU2bs1cO14I7UucTG1H+ozESEohAv2kkqifw9+On/iMCCqcay3LdeO8dbl+U+k1Wzfngp261JhRS7rvVK2e/21gmrCQGmuQd8u5WKE20NPWi0lamD5wn5Nr6WZvs/P8Cf67pL0TlAiXddjEBYGGbIOjeTiIFs3T11gkt+7Nxu2aTsRn1KMYW5QHyoxRY3Wb9zuYAwXQdGuYHxTYa66ZV71eT1gbd5emrewvlbiZ1Lq62QmgK1ShDky9MQA9OeARy7QqldyK/9QVhWljC1gx6sjcGr3lGJigtdGWsKKWLZ6sQFzIisIoE+KXkQ1IQJqBA3Qs5Odu8TbQwDOJZnH6c+k4Fu6EQE5K06urc2omYBOB9fBnfaGjCFkhfVF1dyuaxBL6V38OfPa1JFG49k6M3pVkKvk4oPRfMGmjmIQHdvF/ghkpSOOBXBFCL7XClZmXHCuunl7C+YloyPCm85L5SyPdS5jejwIGzP7JBZPRedu1tCOKwm X-MS-TrafficTypeDiagnostic: CY4PR07MB3095: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 25:glCJwHwjRfAGpnTkQdqUAP36x7W1RzbjzH6UrKOwu2ykZWFQYKQ6j9kdmh7/6+hjHylOugkVyEvMLE0nHyz3eU0qCPDFAXYqVHueTNtgHkpUME1ZlywQ/vtwYDBC3WKLR4sTeb2Xp12SyQzeMfDC1M/ZgxPJAXbpGx5MCKY9v8j3tLKMKHbMtX8wl1b2o3Uj/35qv0sIyncIQojhW79HH+91mMDT5hZ5RjaQyuEryuWiTJC+c9KuJRZ/IkWdyyRqovI1/I3A5WvRqrFfWyxSQwtPwaH3gpfQXP8S/L1aEVGDFnyNgqAN+NDpy2LBi7RF8SSgWOPsUansWmjz7NHJZ72ZFo3zLm2ezSKtwH3m7pHWRQ4V8bHHye9bSgVO+2LHY2PbBUWeGErOcpFFuOAzV2/frNcXSS7th3k4t/36sCTyt4GTUWYecT/ljH8WpPyIgipiASbcwDNka3y9aPJBoCs3dMVWb6H30tHD4iJg0i6rqrJqjgHYVbuDBPUnidZz8pfCeH4wBb//DybmlUFQJAui9QxNtNYrigDvbcCUJJKz7lBnXL30nWDno1rRd+nULKKK/NR5qyKZYqEK+fR2NfWdAkIvYs1nHSVh442U5F6na+lrkRQdk0o0Lq9qOFtIs9pR00/QT5t9J1qnumL2Zit/AkR3ozz9ilp8TOmJ8GMOqNPjnyZ0YT5l6lmILRnE8PeO6EiJVX6V9urtnau+4VtzPyGi1xL03F7nRi9xS4Mb3H/cBvM9s13BDgctk9/tfYBnl05MJlA5QAy/lAwkqPfwruqXMLkmdfzQ/qra9lqNw+sLRwLdqHnJdLK0+61aKFQt4Px9ZFEsD4WIni+h5gW387cTSBNudFHzj/VEBjTMyLTcLnxfhIVshqcGNYt0PbeSxsZzSNOi7cOaUJOIEYG3KtdbJUU6Mpd2hRPdc6o= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 31:URt9nKzV/NDSnb3ZNz6wtEIyhEalunCha4pqb0BsIEtUT/oU6sdxvAQX7QydPMEiG2G70Q89f774kqwl1JwVF7G/fB/kE+Rs99S6AtnlhyJJhrqHaVFzMPX03lJQ9ELmGNNM6RJ9XiaYkDSR4uKUFhBKgB+z55u4O8pvSkgo8fnKcFqCdadqXA7/GDE/HQXF0ve0IypDqK6PuYhnZ0xZHS/3aEZuHx+Hn72vm9nQdz+EW5cFJVmGiKZGnablughCQVfLCe7JHx5YCoQdFxvFjnPdOpN/3kQmcP12roZfia3K+2jNmJC0Xpwvd2FT0QUTcG1+XzD3VLjP6WdnghMCfTYRYgxqKX1SjDgJrIS0FO3GRL3EMAfCflf6blpW5Qje5AG8Z6x21FxoDbQa77kn7A9LNcCK9klNEomluO6UC8wUzc6ZfVLNN9rTzL7i7IIgJ9nWun4xGxYnRSDi1rKAyL1k4MtwEyuAdPOMz4Vagkp68cNOuHYgRZzP4CMeb2KyfjffZOIgWDiNMMKHt0RkDSqA8aUpDaI6L0/8Of5LH5cfhbUWvQUbHunK7qrVCK+d+0OHtwFJGUUOkTpbk5K6go4hiv68waimrHCWvbSEcN9p3NydR1jN8jqjgaZPnUUkr4eosJXuu2NKIci0pd/rHzH+7TlkloPp64hhG4L4tVc= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 20:ugiYRTueyY8IqmRZNQUQY9Ctx2AKZ5f6XKoID1teBo/GpdN17r9MdaFiVR5JUMW8jIym6O0lBipS8o4YLcakwLNvVWcxyHEJJQmcAd4th3gPZ7tz1tHgx2d72FMJTlOGmCYSUWbVD6fmhRhw4j/ybz4uBVyzZw7CHheTg+eXXquASYWg+JaSPsC3v3MZD8bqi+ci9Akon2Tt0WMhfMkLpCoq2GVCHXmbj3tqYx51oojL+cfaSEN01KIBLC+cOjs/04ua4IA/vvrpRL9S/Hr8q48R7dIb8hjgANNqZIeipXuaPOc6LGaMoh4/vBYxxwiPxJEE2HCZKyoxkde/efFivaIn84Q7lMp0TxS1l1MhhdjzMqGYfJMQAfuG7v4etiusaUmrFg5tA2wnJPZ3H+N3WUgpQQdrTK0lbhb9D/eFDPxrSE91hDEIsVZ7gomj0XPil6DK5P4EmwgN2Nt7TQppeubulMxXdGGLP3j9BDQdXRlKu23onjY8f3FfqY6z/g5iEVr2nCQa0/isttjwmnnEz5EoFZR3rayq0ZGrBoY3yoazXDC+9+DkTDyuyAyC9RuDVzBDlYdUjBax18uJS5sLbFj5i7YJTM7l72U6m26aVk8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(125551606395959)(236129657087228)(185117386973197)(48057245064654)(148574349560750)(167848164394848)(228905959029699)(50300203121483)(247924648384137); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(93006095)(10201501046)(100000703101)(100105400095)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR07MB3095; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR07MB3095; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjA3TUIzMDk1OzQ6U0gwNmE5UnRPcmlJYXQ4RW5zR3c2N05CeVJu?= =?utf-8?B?RUNLR2xiWGc0UVp6K3BIMGk1ZGs3eW5UM25ra084cHNkazA0T2pCK1Zockgy?= =?utf-8?B?VGZPODlTN2NhdWpLNy9QTTZBTDU3eU5lRWVDa0xnT2twZ2tWSktzUVhQNEVH?= =?utf-8?B?eXV2bk53MS8vOHVvdVArNFFZTnhyekpVNzVpWkZEaFVNYWZpcG9IYzlJcnZD?= =?utf-8?B?M3ovaHZJbzg3U1pTSlpXWTlhSTNrUU9uV0FIaXFSWXhiMWFmSi9GQldsNE5B?= =?utf-8?B?THpDQUxNc1VsbXhUa1JwY2JrbWsxWHU2RlVIM2lDY1JwdVlQMlIvbXhZR0k4?= =?utf-8?B?ZmlWT01tZGh0K01nd0ZqSlVScWY2NE9vY1R3QzIySWFmc2FHWUZsUGtBYnFa?= =?utf-8?B?YnhEeTVVZTBJRXh0ZjFkK0RZUXdDeFRPOWloZXlmTENkOE1QK3BpTENGamNG?= =?utf-8?B?QVFtVFo4UXNpa1k1d2ZaZi80c25tU1ZEalBmZEVPSDdRNnJaN21FQzgyOGJm?= =?utf-8?B?TFByZlpXTWNCd2gva0FmdmVrRDdQMnFRQ0tiOHFyS0l2Ykp0U25McW5PNlQy?= =?utf-8?B?OS9CRFo1QmJ4RTVCcHg4M2xxN0hzYTdQelZYeTBCSldlVWZ6WXBZT3licS9n?= =?utf-8?B?N01VWGZMTUR1VHVHcXFQQ21UOXJkVlJFQ2dpKzVkc1pva3I5MXZaWkdjQjk5?= =?utf-8?B?QU03d2kvYys5NFE2TGZFRzFyWUo0aVRHRlVpNTBOVGlYS2dZazU1Q1pObVhD?= =?utf-8?B?NWlYQVJxaytHM1N0TWVRM2tKSk1TZlpodkh4TXFoc01PakVQb05jOTdSdnFK?= =?utf-8?B?c3ExVFRkc0wzdy8ydFJndmdPSWoyT0dpMzdTSmFJZDBHWTBEOXRYTlJIUmpX?= =?utf-8?B?UUN6Q2JZOVBLVHQvaFNOU0crdjVqNkhSNElLOUFiQTZPSWs4MzExME9wZFJL?= =?utf-8?B?a0JzMEVBTzhtVGo4QWl6eUJyWlBMTE9IZFJjL0QwaFZ3a3BwZHFRMENUbHZS?= =?utf-8?B?M0tiWDA0Y3h4S3lSeHlwejdySWdpc05PeEh2RDgzbmwycGxnSnVuM280SjJC?= =?utf-8?B?V25YUXRaRlEyOTdPdjlQeks2Q2x2UGpRMkxrNnhxRWFIZWRVWWpZMnJONGVY?= =?utf-8?B?dS9BTXlSeTRGVzFVdkFhdU5tTVN4a3FpY0lvVVJFUlRsS25PeEhnWUFUWjFr?= =?utf-8?B?eDRiYU9CcW1xVHNvVHRKM1RpYVcwM2hIOFVpZUlHQ25ab3ZmN1I5bm1zRzNS?= =?utf-8?B?YnhMT243ZzduOWR2NllGc0ExVFB5bDArYnhuRzFZakxidW4ybzlseFFEWi9x?= =?utf-8?B?eUtpL2FVWG92Q2hYemdndzlua1JEa1FLU3A4WEt1U2FwNUpIYWpQM0tpOGxt?= =?utf-8?B?SDEvZEVNT1A1Uk0vWnhoSDZhM2thQ3ZVamx6eGpGVm1aK3RtVWVya2lIOUVP?= =?utf-8?B?R0Y3S3ZCS1FWUWhycjVxZ1lDaDdWcFpia2ZhRFpZZTNzK2srQ2kzRVdYdGJG?= =?utf-8?B?b2Rvc1dOL2pYMU1NTmFyV3ZGd3huN1k1Sldyd0FFU2xQWis4NUE0MHlSVHFZ?= =?utf-8?B?UHpGODZ0Z1JQMVpNQ1JQcXAyVEZwcnBTdmlvQXhoUFh6SzZnVEZVN3REUVBa?= =?utf-8?B?b3RqYzhiMHM3Y2lBcW8wOXViVVVjd0J4RTZ1dVBkOW9RMHpGVHpNcXZoSFhx?= =?utf-8?B?Skp6emVEall1RXkvb1RqN3A0cmtyM0pvb2xBRTlsOE5IY3oxVTdIRkxkM3Z4?= =?utf-8?B?U0ZXNmQrR3Q2TzdxZXJoZytnNFNmaTVJeThBWUZrYnQza2tQR3QybSs1MFlt?= =?utf-8?B?OEMrbUQ3QmtNOVJPQjVYQmo3L0JyOGtqay92RUNhQ0FqcmlZZ0k3VGdsa1Yx?= =?utf-8?B?RE9mbldlc3ZMUXR3Q1ZKUitDQWxDdk45d1V0WXV1TFVtT2RwcVYybzRCU0RE?= =?utf-8?Q?V8gT8+krxk3y3ce4FF8HPiNaJjzHw=3D?= X-Forefront-PRVS: 03607C04F0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6049001)(6009001)(39840400002)(39410400002)(39400400002)(39450400003)(39850400002)(377454003)(51914003)(24454002)(86152003)(65826007)(6116002)(7736002)(83506001)(90366009)(4001350100001)(3846002)(31696002)(23676002)(2906002)(229853002)(6246003)(50466002)(33646002)(6486002)(189998001)(1720100001)(230700001)(6666003)(4326008)(36756003)(25786009)(76176999)(31686004)(305945005)(54356999)(50986999)(93886004)(77096006)(42186005)(81166006)(5009440100003)(5660300001)(551934003)(66066001)(5890100001)(65806001)(6636002)(8676002)(38730400002)(65956001)(53376002)(42882006)(8656002)(478600001)(53936002)(72206003)(47776003)(2950100002)(966005)(53546010)(6306002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3095; H:[10.160.0.43]; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjA3TUIzMDk1OzIzOjd1MUQ4Vnk0bUFJUFR3NXRhOEhxVjVsQVEw?= =?utf-8?B?NkQ3Yjhpcm5tWlpvN21HMUpxTFpqRnRBRHluK2c0Njh2UnAzL1VKYVk2OHV1?= =?utf-8?B?bGpUejZoTXBkNFNrYUhPYkc2clhuUWQ2T0kzVGliUG0rcXQ5dVZZMCsxNm50?= =?utf-8?B?UlAyelZXLzdTZDNrd292bDZQODRTQ1lVdXdLUnh2ZGZHVEgwTExPT2UrQlZ3?= =?utf-8?B?TG1KK2orTXRoQlhUUHVpWmlCSVo2VkJBSmljZWw4R092bTZuTEVNZS9tNzlF?= =?utf-8?B?VXNhbnkzY2kzS0duR2dGSFhhbXJOMGRDdVlNT3dLc0Z2MjdUODRpVlpESU0y?= =?utf-8?B?YVRVd0M3QTBVdWJxa3k4a0IzdU9tL09YbGlpOVpaNjM4N0UvdUl0Z0N1U2xM?= =?utf-8?B?QjdTQlUxdGFFbENQVjROTWZ0QnQxdmUyU1dOYzBPTmxxSCtmMXZhOHo5MHlB?= =?utf-8?B?ckdIVW53Nzl2NVlNQ3gvNnZuT293U1pHaWMvTEFyOStnQnpMT3ZoWVhDaFJE?= =?utf-8?B?VHBKWnVmUkZMMVBaT3Y3UzdwV3pra2JyWG5ySFU2aVFxcjhUWkp5cWNzZm1s?= =?utf-8?B?SFk4YXVEeHNkZGN0MktneVJlWm0rZG5sQnlDQUh5Mk15RFFxN1FNbG1NSE01?= =?utf-8?B?KzJ6ci80QnJhUnI0SEtTbmx4NlpyVFN4TlZBaDhTUW8xcmc4UEh6UGgxb1Nx?= =?utf-8?B?eVQ5akMyWnhXdnoxMDM0SklUOTlCN2NxaEh5SmVuTm0zSGlFMGgxd0hheE1V?= =?utf-8?B?VHIrQmlUcFRVaEh4YkVyeFdzYyt4RDhYR2ZINDk5MXphYkNwUWxGNklRZXFR?= =?utf-8?B?Qmc5SUoxYUFrUVNyUmxIYTQ5b2xEUUV0VFBvNTVYY1RCdkxJb2QwR2pnS3FI?= =?utf-8?B?Rlg1cS9jK0cvNXpwekhITWNSeGpXanB1UUk1UEJTN09FVFljaUxrTk5CQ25n?= =?utf-8?B?N0g0OHVqd2VsdjBpcUJtV2lXT1EwS3pGbUM2MzhBdW16elRPUE1jajlJOVZD?= =?utf-8?B?em51aUZDK2VOUTdlNVh2eFVyYisxVTlOU1ZqekV1MUNFRGU5VFNTaDFjcEV3?= =?utf-8?B?MFZIUUlrT05DUjNNbVdDRmgyNytZQTRHU1IreG9FU1gzVXlOMnR5WUxFZTR3?= =?utf-8?B?Tm9PNXNCV0FtdytMV2JBaDNQeXpJQVZFZ2cxSXRuczNPRmd6SzBNa2FpNEF2?= =?utf-8?B?eHZUUDRlcHJRaFZnNkVDczAxSm9veWNBNTdJT1RLdjlWM3BiYTdCL0c5aG1N?= =?utf-8?B?Vk5BOGcyVWxoT2F4d1B3R1c5MWc0UU9ydm9RR3REb2drU2dRNkFvLys2MVJs?= =?utf-8?B?UDUzbklLc3luZGUvd2E3dlduWDFtQXJCNTgybEdMRUtDQmNTdFJ1elhoS1pi?= =?utf-8?B?Y0NQWlNUV3hNLzlpUUtaU1RydENEOVlJZEhSRmsrcVZRQzRuT1NYUTlBWkNK?= =?utf-8?B?azQ0eEY2dXVrRlhlOHdxK01aOWdCNXU0UUZWRzAxbFQ5YlVQUGZMUFRZSW1r?= =?utf-8?B?Vk1YLzJjblBEc0FQbklTTU8xUFpmc0lKeUFpZGZOSFBXVlpjMUJPN1BSRVhX?= =?utf-8?B?MU1kaCsrQUlYVmpxSklPeDY0Z1I5djlZZHNhVmJOQnBYc2JzbEFERE85WUZK?= =?utf-8?B?RWFEd0JhSGtGYUdia2pWRUFGVyttVGxOQ3kyK1RnRTFod09zTHhVZGh3b0Yy?= =?utf-8?B?ZDlkcFM1ZEdvbVFHSmtianZDY2RSdE1XTFZuNGIzaDdBbWl4Q2VWd3dNOVRO?= =?utf-8?B?S2RuV29ycXhrVzQrVFR4anZOSWZYVWdDKzkwVUQyOHJxRVB1WmVkTU9mdzNj?= =?utf-8?B?Z3hMSFQwelB1eVZERWVZaDk5OW11VmhicFozc0xkdGIrcVNOL05LTEJqMDMz?= =?utf-8?B?ZlV6cmNnc25lbXBiL3I4enNoVllkWDMxT2FkZUJ3aEtTeE5lWEdZZHVmMnFv?= =?utf-8?B?UnJjZmdERG5qZllXT0l2WGRYRTBKa1lUQXpkWkNjVkxXWmlYM1hsSHo3MFlO?= =?utf-8?B?MG10V0hBOWgrSE95b3Q3ekxscmc5MUJ6VVJZSiszazVqMG9UQmsrZkpMa1Qw?= =?utf-8?B?ZnRJVHdUeXY2YkRZdEpqNjZqK1FXTUxkQTBqc0ZyUmlKdEVQMGRUSTlveE4z?= =?utf-8?B?WXc9PQ==?= X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjA3TUIzMDk1OzY6OGRqWlFoN0VlUHphb2ZzMjdvL21GYW1kUmNa?= =?utf-8?B?NlIwYmdqZlA3V3hmaWZINjBSMjZEZ0xnVVNWczR4K0o2bmR1Q2cwc0orQXJX?= =?utf-8?B?NjhzdnBhSjA4RkdXcldwOGcxSlJCZGl0K0NKRVdNSUZKbE51RDdaeW9TOGpL?= =?utf-8?B?cHZiVzBpdlY5eFlWbjhZTEI4WW10OGNFRkl5bElyTzF5cVFDSlVsWjdmZnZl?= =?utf-8?B?SmNVWENJWGphU09uVU14TWNYMmlzY1ZWeXVBNHZSREJTa0l3Z1BURXMzNzRP?= =?utf-8?B?TU9wSXJNQWd3UW5CQkcyVkxkZVRmTXF4NzBmQ2pyRkJzYndVZHdBYnh0TEU0?= =?utf-8?B?QTN1ZVF1VkRpc1RZSi91bm9GWk1GN1BVbUE5dVYxVEhXZDZENnVHUVp4aWNi?= =?utf-8?B?TWNZRm4ycDhjSCt3YzNQOTg1M3dQUTJVcGR2NVhKaWJyL29ianVLTWhsUDk4?= =?utf-8?B?WjFtYi9BamZ0aHB4NWF0ZFJXUTd1SldaMjJQWXBkQU1UN2gwSWVWRG91L21B?= =?utf-8?B?MWE0YzNqNTF6czYvK0xuREZLZ0t1cmpNM1REUlJqMHBiVFkrK1k1MW1QTUV0?= =?utf-8?B?aEdLVUxQdHhmTVRqcnNSNllCVWR2SHpVZXduRi9LOXNwRDl3cTU3SnZzSDdy?= =?utf-8?B?dHdjRis3VHNmSHBoR25BSC9uTTlYV1lCRTRJT21KMUl1eHd4NlA0MGxlUXBM?= =?utf-8?B?OEMwd0xTb3IvbFVKK0xBRzViay94cVozcG5NSHdpYS9IblJZUDNBNWlhRWVj?= =?utf-8?B?VXhwOEtMSnhZeEhtS3RKQW1zS1kvb0ZhWExkbTZoalFWVEQzZEczUnRBNVpa?= =?utf-8?B?ZVNSQ3BibkdLM2ZjQjlxaDZBQWpONXkzOGdtRjg3MjFleExFZE00cVpnY2Zu?= =?utf-8?B?K3hYLzkrMmliUkVFMkxGd3hmZ1ZTTFd3Q2RVQ0lJQzU4QjZjb3BpZUVPeEp3?= =?utf-8?B?NTcybG5rdDI0em11V0EyaDNrT0FJTTQzSDQ2SUJQRGdPd3IvdXZwbnRram94?= =?utf-8?B?QTlISlFBZUNhTDRVT2VzS0ZlMW8xMkt3UnZhcHowZkN5R1puSTYxb0cxbWdW?= =?utf-8?B?a2c1cGJKbDNiZVpBS2hIS1VQMEJ2L0pGVE5nZEdUbnlSSWlPWHE2OFVvWVpj?= =?utf-8?B?U0RhcGM0TThLQmNadEYrMmZxa1hpb2RtNFQ1Qlg1VkM1aEVnZGtrY25HM2hB?= =?utf-8?B?UUV1SGRUVmRhOEEvS1RIa21oanpnWVkxclk0R29kWGE0NWNORmNzVjQ4dEdo?= =?utf-8?B?eTNndnMzZlUzdEloeWNENDA1a0p1Y2lSWGE3ekVwbDQxTExjOE4rSGpUeDBE?= =?utf-8?Q?qKK1+w0MUuYYr7uQ8yKfMI+pz0tDXBc=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 5:8TY/UfyPDW9Jy6NVqPJUSG8Wjws2/5RsxK56wabmGjn7ATDbb0em9g5mnJ+wn9RHOCQV9MzCXAz2NHnDEcNtkUJ79f5indPYaR9hm5GM6eUWuBYOLKj9BINjEJ6MONRSRpNHqW0NBY3aYuiiNsGyUs4HPVYkgyAO0H7BJbdXHrmd1TSyyrnRex4LENpGGI7jGIsHwdI3Dgmijw+btCNznVMw+WoeHk/CNnh3YKHbIEQ2rzpAPxbWCbWLPopeYC3bdb4gr95vMvPHv490KJLVKhL1KwovUm/AdX9CzQugMqlNXQ85bIA+QVDNqVLhqz8uyK499EYJniYM5WkDKectdLf5BtmYN5fFNpioBIgrNBT/+r9R8PRDVp+QjBtRfg2e64BmubEwL4/0DCzSZ9umxOISkEzAakCOG3ioflJYDPFwQ3ARSGiVL22xFgYyer27qzpUStHBV/66OArCbyHVpPtAJCDymeBFAuBtJ2gkeb4b6mULtJxTYZTUpgZt3m5k; 24:gTR8Ml1kR/KiRMMEaBRyTFTJTziZnwn0ii9hYbX07I2q1HdBAjQSwvl3eqhRxvQcu8dy6EfedxEeeQGB/CbPvwUKSxnywfAQb+81DvInyuc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3095; 7:iMNVKelVUEPSGUe9HE33vi5tpUYnew1eK7ZIQQ1prxkmOFS4ZFQXqAR+lBrZ6Whq7H7w8EyBJg59JBlrPK2oYIkAzdlWWIJK6z8H9KewhHEvVqjuZc7iGQVjsysTPt+5BMrwpKAMEWRVTbIcsjJIBhR4dyyu3+oFxpj2YhYCBYobEHY1cqwv08/VX8kuJ+r788G/zbDgrK0OWFht/jqOOVpJxK6guLzCkHMuoyXoqi5Z1L6Dyt0981j+8XoxfEGv5b57mgwonwZhtxPYDljaEuQFmnORrACvN06+RwAalUIPTOGLJ/zvaQQ1vaslGhfdRPex0rj3qxQsJRcbPGh/SSPBVtgWfsrKTNVxwqDEiMqVUjKMPHfGr/lFekK8wiiKg9uG6YBhRrupbcj5hLH7h8XuCGW+v9qMCpkkgVDoCK68uqCVZDEMz5V2Omm3rNLr9UOmySalaZ1lBQBz4Q5iT2iyi6anqmaE6ojkdR/yXS/8AOvqcLJKocVZOKsBXFw+xQjszpXDNMO1d3i7dnvEYWGfo6uNtUPxFaO/vr5hydaPGepseJvml9Qfrkzwr9sZuUVBcGQR9gFVHTQsiejsT+MI9DgoQY34zGAqn6kjAesam14V2VNdtpr2BT9QIbZ3ioZ3/Juky85JXtW2pkNHVkbTv2Hv+Zu1NR861vQltkZjAyvOsCdMTBCLh1UgZUNwTThogkcNdGEEQDHumhhU2kLYKpKA4zi85V/AmaWe9q1wCBlqhoqoAKsmS3doHM25RPGUFPG+kr7RCk7bVT/R9J3oIZ+cXUplFPWw4RG85QM= X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2017 11:19:20.5903 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3095 Subject: Re: [dpdk-dev] [PATCH 07/10] linuxapp/eal_vfio: honor iova mode before mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jul 2017 11:19:26 -0000 On Thursday 06 July 2017 04:29 PM, Maxime Coquelin wrote: > > On 07/06/2017 11:49 AM, Jerin Jacob wrote: >> -----Original Message----- >>> Date: Thu, 6 Jul 2017 09:58:41 +0200 >>> From: Maxime Coquelin >>> To: Jerin Jacob >>> CC: Santosh Shukla , >>> thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org, >>> hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com >>> Subject: Re: [dpdk-dev] [PATCH 07/10] linuxapp/eal_vfio: honor iova mode >>> before mapping >>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 >>> Thunderbird/52.1.0 >>> >>> >>> >>> On 07/05/2017 05:43 PM, Jerin Jacob wrote: >>>> -----Original Message----- >>>>> Date: Wed, 5 Jul 2017 11:14:01 +0200 >>>>> From: Maxime Coquelin >>>>> To: Santosh Shukla , >>>>> thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org >>>>> CC: jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, >>>>> shreyansh.jain@nxp.com, gaetan.rivet@6wind.com >>>>> Subject: Re: [dpdk-dev] [PATCH 07/10] linuxapp/eal_vfio: honor iova mode >>>>> before mapping >>>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 >>>>> Thunderbird/52.1.0 >>>>> >>>>> >>>>> >>>>> On 06/08/2017 01:05 PM, Santosh Shukla wrote: >>>>>> Check iova mode and accordingly map iova to pa or va. >>>>>> >>>>>> Signed-off-by: Santosh Shukla >>>>>> Signed-off-by: Jerin Jacob >>>>>> --- >>>>>> lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++++++++-- >>>>>> 1 file changed, 8 insertions(+), 2 deletions(-) >>>>>> >>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c >>>>>> index 04914406f..348b7a7f4 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c >>>>>> @@ -706,7 +706,10 @@ vfio_type1_dma_map(int vfio_container_fd) >>>>>> dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map); >>>>>> dma_map.vaddr = ms[i].addr_64; >>>>>> dma_map.size = ms[i].len; >>>>>> - dma_map.iova = ms[i].phys_addr; >>>>>> + if (rte_eal_iova_mode() == RTE_IOVA_VA) >>>>>> + dma_map.iova = dma_map.vaddr; >>>>>> + else >>>>>> + dma_map.iova = ms[i].phys_addr; >>>>>> dma_map.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE; >>>>> >>>>> IIUC, it is changing default behavior for VFIO devices. >>>>> >>>>> I see a possible problem, but I'm not sure the case is valid. >>>>> >>>>> Imagine you have two devices in the iommu group, and the two devices are >>>>> used in separate processes. Each process could try two different >>>>> physical addresses at the same virtual address, and so the second map >>>>> would fail. >>>> >>>> IMO, Doesn't look like a problem. Here is the data flow >>>> >>>> 1) The vfio DMA map function(vfio_type1_dma_map()) will be called only >>>> on primary process >>>> http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_vfio.c#n359 >>>> >>>> 2) On secondary process, DPDK rte_eal_huge_page_attach() will make sure >>>> that, the Secondary process has the _same_ virtual address as primary or >>>> exit from on attach. >>>> http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_memory.c#n1452 >>>> >>>> 3) Since secondary process adds the mapped the virtual address in step (2). >>>> in the page table in OS. On SMMU entry miss(When device >>>> request from I/O transaction), OS will load the mapping and update the SMMU >>>> "context" with page tables from MMU. >>> >>> Ok thanks for the detailed info, but what about the case where the same >>> iommu group is used by two primary processes? >> >> Does that case exist with DPDK? We always need to blacklist same BDF in >> the secondary process to make things work with existing DPDK setup. Which >> make sense as well. Only primary process configures the HW blocks. > > I meant the case when two BDF are in the same IOMMU group (if ACS is not > supported at some point in the hierarchy). And I meant two primary > processes running, like for example two containers running each a DPDK > application. > > Maybe this is not a valid use-case (it is not secure, as it would break > isolation between the two containers), but it seems that it is something > DPDK allows today, if I'm not mistaken. > I'm not sure how two primary process could run, as because latter primary process would try accessing /var/run/.rte_config and would fail at this [1] point. It's not valid use-case for dpdk (imo). [1] http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal.c#n204 >>> >>> I don't know how frequent it is, but if ACS is not supported by either the >>> endpoint or the the root port, then you would have to share the same IOMMU >>> group for all the ports of your card. Right? >> >> ACS is supported in our card(it not in bypass mode) and one mempool PCI BDF >> comes as a IOMMU group. >> >> If it in bypass mode anyway you use in vfio-noiommu mode as >> there is no protection anyway. > Yes. > >>> >>>> Let me add the background for why this feature is required in DPDK to >>>> enable NPU style co-processors. >>>> >>>> The traditional NICs the Rx path code look like this: >>>> 1) On control path, Fill the mempool with buffers >>>> 2) on rx_burst(), alloc the mbuf from mempool >>>> 3) SW has the mbuf in hand(which is a virtual address) and program the >>>> HW with mbuf->buf_physaddr) >>>> 4) Return the last pushed mbuf(will be updated by HW by now) >>>> >>>> >>>> On NPU style co-processors, situation is different as the buffer recycling >>>> has been done in HW unlike SW model. Here is the data flow: >>>> 1) On control path, Fill the HW mempool with buffers(Obviously the IOVA >>>> address, which is PA in existing model) >>>> 2) on rx_burst, HW gives you IOVA address(as address as step 1) >>>> 3) As application expects VA to operate on it, rx_burst() needs to >>>> convert to VA from PAA. Which is very costly. >>>> Instead with this IOVA as VA scheme, We can avoid the cost of converting >>>> with help of IOMMU/SMMU. >>>> >>>> This patch set auto detects the mode based available of type devices in >>>> bus and provides an option to override mode based on eal argument, so we >>>> don't foresee any issue with this approach and welcome any alternative >>>> approaches. >>> >>> I don't question the need of the feature for these kind of >>> co-processors, using VA as IOVA in your case seems very valid. >>> >>> What concerns me is that we change the default behavior for all other >>> devices. Having an option to override is fine to me, but the default >>> mode should remain the same IMHO. >> >> Doesn't seems to be a technical point. But I agree with your concern. >> we will address it. >> I think, we have two ways to address it. >> >> option 1: >> - In existing patch, >> a) we are currently setting(internal_cfg->iova_mode = RTE_IOVA_PA) >> http://dpdk.org/dev/patchwork/patch/25192 >> b) only when with eal argument sets to RTE_IOVA_VA and then bus probed >> value == RTE_IOVA_VA the final mode will be RTE_IOVA_VA >> http://dpdk.org/dev/patchwork/patch/25193/ >> check the code after rte_bus_scan() >> >> option 2: >> On rte_pci_get_iommu_class() in http://dpdk.org/dev/patchwork/patch/25190/ >> we can check the rte_pci_device.id.vendor_id == CAVIUM to select the >> mode so other type of devices safe. >> >> I think, option 2 makes sense, as it gives foolproof auto detection scheme and >> without effecting any other devices that not interested in this scheme >> >> Does that address your concern about the patchset? > > Yes it does, or maybe create a new flag in struct rte_pci_driver's > drv_flags to provide a hint it prefers to use VA as IOVA? > > It, of course, would just be a hint, and should be set only when other > conditions are met. > >>> Wouldn't it be possible to default to VA as IOVA only when an HW mempool >>> is in use? >> >> It will be too late as in the normal scheme of things, application >> creates the pool. > > OK, makes sense. > > Thanks, > Maxime > >>> >>>> Similar problem exists in another part of the code in DPDK, >>>> http://dpdk.org/browse/dpdk/tree/drivers/bus/fslmc/fslmc_vfio.c#n231 >>>> Its a conditional compilation based approach with duplicating the vfio >>>> code and we are trying to fix the problem in a generic way so that >>>> everyone can get benefited out of it. >>>> >>>> Comments are welcome. >>> >>> Thanks, >>> Maxime >>> >>>> /Jerin >>>> >>>>> >>>>> By using physical addresses, you are safe against this problem. >>>>> >>>>> Any thoughts? >>>>> >>>>> Cheers, >>>>> Maxime