From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <hemant.agrawal@nxp.com>
Received: from NAM01-SN1-obe.outbound.protection.outlook.com
(mail-sn1nam01on0079.outbound.protection.outlook.com [104.47.32.79])
by dpdk.org (Postfix) with ESMTP id AB98B374
for <dev@dpdk.org>; Thu, 29 Jun 2017 07:32:37 +0200 (CEST)
Received: from BLUPR0301CA0020.namprd03.prod.outlook.com (10.162.113.158) by
MWHPR03MB3280.namprd03.prod.outlook.com (10.174.249.36) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id
15.1.1199.15; Thu, 29 Jun 2017 05:32:33 +0000
Received: from BY2FFO11FD030.protection.gbl (2a01:111:f400:7c0c::162) by
BLUPR0301CA0020.outlook.office365.com (2a01:111:e400:5259::30) with Microsoft
SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1220.11 via
Frontend Transport; Thu, 29 Jun 2017 05:32:32 +0000
Authentication-Results: spf=fail (sender IP is 192.88.168.50)
smtp.mailfrom=nxp.com; samsung.com; dkim=none (message not signed)
header.d=none;samsung.com; dmarc=fail action=none header.from=nxp.com;
Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not
designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com;
client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net;
Received: from tx30smr01.am.freescale.net (192.88.168.50) by
BY2FFO11FD030.mail.protection.outlook.com (10.1.14.211) with Microsoft SMTP
Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1199.9
via Frontend Transport; Thu, 29 Jun 2017 05:32:32 +0000
Received: from [127.0.0.1] (B10814-12.ap.freescale.net [10.232.133.65])
by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v5T5WNWi017061;
Wed, 28 Jun 2017 22:32:27 -0700
To: Ilya Maximets <i.maximets@samsung.com>, <dev@dpdk.org>, David Marchand
<david.marchand@6wind.com>, Sergio Gonzalez Monroy
<sergio.gonzalez.monroy@intel.com>, Thomas Monjalon <thomas@monjalon.net>
References: <1498553186-24541-1-git-send-email-i.maximets@samsung.com>
<1498559080-27331-1-git-send-email-i.maximets@samsung.com>
<CGME20170627102451eucas1p2254d8679f70e261b9db9d2123aa80091@eucas1p2.samsung.com>
<1498559080-27331-2-git-send-email-i.maximets@samsung.com>
CC: Heetae Ahn <heetae82.ahn@samsung.com>, Yuanhan Liu <yliu@fridaylinux.org>,
Jianfeng Tan <jianfeng.tan@intel.com>, Neil Horman <nhorman@tuxdriver.com>,
Yulong Pei <yulong.pei@intel.com>, Bruce Richardson
<bruce.richardson@intel.com>, Jerin Jacob <jerin.jacob@caviumnetworks.com>
From: Hemant Agrawal <hemant.agrawal@nxp.com>
Message-ID: <b4aafad2-4636-0b18-b818-69c52d0d59c8@nxp.com>
Date: Thu, 29 Jun 2017 11:02:22 +0530
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <1498559080-27331-2-git-send-email-i.maximets@samsung.com>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-Matching-Connectors: 131431879527434696;
(91ab9b29-cfa4-454e-5278-08d120cd25b8); ()
X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI;
SFV:NSPM;
SFS:(10009020)(6009001)(336005)(39860400002)(39450400003)(39850400002)(39400400002)(39840400002)(39410400002)(39380400002)(2980300002)(1110001)(1109001)(339900001)(24454002)(199003)(377454003)(189002)(9170700003)(83506001)(189998001)(356003)(498600001)(33646002)(104016004)(2906002)(38730400002)(5660300001)(2950100002)(50986999)(76176999)(54356999)(305945005)(4001350100001)(120886001)(85426001)(23746002)(7416002)(54906002)(53546010)(31686004)(64126003)(105606002)(106466001)(86362001)(31696002)(229853002)(93886004)(36756003)(65806001)(65956001)(53936002)(50466002)(47776003)(6246003)(77096006)(6666003)(81166006)(8936002)(4326008)(65826007)(8676002)(7126002)(7246003)(230700001);
DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR03MB3280; H:tx30smr01.am.freescale.net;
FPR:; SPF:Fail; MLV:ovrnspm; A:1; MX:1; PTR:InfoDomainNonexistent; LANG:en;
X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD030;
1:s/4pi6hfMeeemb7d1xcSqt2uX/tmAU8+efxusvpL07ouIPauw1pwoh7cpeey2FNtdmnYyLgKbbaa0sI6J4Bo02EPbYrbWX4LhLmGmiWlRq8LNmCk3SuT1jRoHEt5ULxp7O5TRvbPSWqgktQRK03CInQd3EN4dNnnw5uCSWikERPXKK3JAsrS9Ke5pB8Ebsb/KHG4Xy8ynhtWtNE3XIZDWfseVkW9wE8wqGpnbpMagnWkBPkgaTU0FYM2xrz71BTZJMZEDgAE87ZAQ5bOQ27DKgUeTsj7BrwGETi4rar2+hi7RXHBh8kqPGWb/YaGPkcK7AXafF8kcc2URcLMVyOSlB7fKKsqtDewn+CZa0WI2ysTV3LQGVd1P4Le28jDfsVHRM6lEPzJ8GdtwbOjW9yGOefoarw6QXRru0101xWxY0cLzCYyDDem+w6CX4YtXafgyqGf9NSPVU4rwT6CtlMXSvQmiNBQZwx6UqbhVGLd5m6qE/JDfrAGKtkPSQzXq5GRMYxs3iSbeYAzpxp2lKdJlnBJyDlnt6wv5FF8itc5k5sNwwD1Aioj0zbH6ddl0G2zh2UaRXj0h+8PRgjk7j/YtBSpEvowigKS8il1EshtgyG7+EQKtNGoOCAZW3hket2DdKpomut0llZTt2eMvptVhA5mVdDLiSkNDnlDR47M9CKJ0bnklg37QkH3s1V/Al2m8tY42AMyYYAM0diJQ4GA+YMK3Y5sK3EUnkjf+jz6I3vKL/DQF7R0zES++OD+1p/nmmX957dsdkZgKoJeBL6C99ANbiG/h2s/3AUriu+YULzncMMLVcCId1CiuP5DGAY3C5cUwkCw1f68NnJ3KAVjxkiN+H6zszAsaoIPYH2SdEmdSiAjZdsf36RFhjdshp2/SQ8zdNyvZxr+81vy+0Fk4M1k1eB0uQ9WWCggY0xo+ipO6E+Q+R20+6bM0LuFa+hk
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 48750c5f-df82-495f-a867-08d4beb03dd9
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0;
RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(201703131430075)(201703131517081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);
SRVR:MWHPR03MB3280;
X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3280;
3:Q/b9iC8G+RQnzAs6t/J2FGf8XZKHt/vWg6P1HqXQLx3JKvvylvHLdK1UEVZi7AUesP2/zBh2kQQeLu7Hp3ThLqnitLR/fBKuO1bIxnzcvxBpQn1QPi+arrtwKA89S58xHzfS0aD8iLYIrfm1zc5+Jg/mb/WUB/s5gwha7FDeZQgs7p4QkjONpsCMKcZJtNYiIz4A5fScLczD95ev+UuhZ4EbMHhDky6qRui/x0cFN35fuqCwGWWmpP2Ogij9XZHOq5iShe+ZTdrOqF4U5aPOB8L4wV4rhlbyOhltZ8Shvtwj4YaRJwhLCrwq8Vc1pUkPPLSRwf71paZcQsCEfc3f5ZiRIOruwQpn6B/LylTU1t6kcM+WPYhb4XY/4x5oVrgpG0fNsjytFHjYTgo0iEQLp7UIDB1ESojLASkrMC2AMwytBQ9u2NeEGJcboWQgAqVo00niP/K38ahRt373TBoaN2jdk2SkbHMbTpJ+tf84oQzL+bGc+oWvNiPEJ2JOc3srsDwRvMXVW16kSHMv7Vs3pdBY4xVvS3DgmT29b//x4qF/6uqUIiM/dLhZtbEvn47mFQHlIginS2txHmh6wXaFGdZwO5ux1Nq3dc7k13CI0G8y5t0HuLYTv3n7koLda3J3UMeu8suwAGd+04HHo/s1UkN2eMf4XvKq/iSUFFi5q9o4mEsZEwP064zfmHGv4hkwJlWljqa0XVneBmtHbpJ6uN3ZpxoUOz29DmDzb7cl/QAavqNUZ7OMz7XidRyQ/4DX14hPxQPP/S2gex6WZlkRRD8VqhXVwo89xS9LPom13kuBCvnnnw7IY7hFrB7u41F5IlhfYe/EAInAHD2fUZiLHw==
X-MS-TrafficTypeDiagnostic: MWHPR03MB3280:
X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3280;
25:KiP1hhsABQdHcNtOikZGEiA4ecLZ/xCY/ekoHef4t6aTifsWuDmsY9bZ7kRwAqbcMRIC8G/KqDt4imKR/dJ1IlMH9yjR5moXprcTY5PDzsMWbY9t706GKZIKYIFzuzjDWKXaWGEJn4P7SsnTFtl5TROs9lTSKA+g5LyHuauPJCx/XPx9oLHi/ts7kak5eTE2Sa4ic9dAZqHrTHZXwT57gRdPfm4EzMOwfgmyb4nuQfC7+HJg0W5mGJxbgPpXJrcUHwtaPY6d/RzSL6CjSjAwjHWzWLEsPTLebfaCmBhUndwCzoP6UjyiDhFPRvyAf+EVstZZt59me3pq3eykbZZCCGRNqDOPw2bDOyoYPQ7nULQoZfQ47l9qiklg6/iceDaC68535+aEjnYgRr/DJWOWa9A5Fj+uvtCb3UI0ZZYYEU2pfRV76LxcaIdCC6OaL7f0OC49MCDvadinztYTADkksx/UIwJtCkM2wFGnHUaF/pnzstL0n59D4nEeq0R4anbf0nWb6J6ZpCmqjxRzV6OLYvKPJtw7nI8Piue5qNDHIo6mdlwJ5aqTaJXK9l1DpUoMfziXI3YYHuJyPzIzuQMhrOSwuGRcL4EE9+JmT3oCmgDzOul6Qy7fPcyb/XtIypKGKipe8CvHj7uZFcyHg6irTJy8OeXAi8LccMwwXILuTMdI/oYb3zePgbgR56ieYZHKtv2CT0VWFwbzt6cMRSRvDi9C9l5mMfpusvS0SeVLC0SbbZI/b5iBhjPiS9Tdgp2w9cXaEyJvJ/dtgchA9LE50RmmXn3VV2R1A7jvSzRTmNcxfDdAGtXDtCGIieo3qia4RywXRXbEzAdsdW3tTkCVcJsptg3I0jAMWfBDOWdLsOCXuk7xTYxJ3BWL66gUbYlERoHcmK8Xn4aK8iKrDePrs6IiC36Wtcfj4Lk725G5a48=
X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3280;
31:a3P3yNVx3DM2VvCvPhifwKMpRkRmD42yuXHKodiDqqlUqZ0xx5wH+diEeYiLPlKilScSiDC/6yNQYL0Dov4tpnUQ7fMK+SCw+eI1YGi2DUNf6YdGM1yp5qb0vFX+Jyy0LC8tEz+0nu1Z2mYHuru+uIybUFRMLTPuOdr2hYF4sxhKa8tiYGoerUOOoyOx1dJDG+VATfrb6hVtSQYd1I3scvjKxsS1jAyKnLuYrZUNwrr/sLt0jFFC5I1Ns4AJBqHgxrCgH5gTGPCOPR6okNGQHNkm5FleFvl7nVuFA4l1cbSXBWhkWO4b6Ur5ci603jRDS8uIy/ZqPD0wwf8XLV9ZIZGe9oFHu5R6q4/wATraZ62YkG83zSBFO40h42p4moQHDTb1Y3hNK8FIUtPI8U1wPAgiF73uSaDq3qbKJYZ3+upxNUKvK/3Xjl6CY9n4vncHLxLb27UGJQPR3agusfjJ8TsXuc2QVNzNs6OjG3JNC+N2N9+HXClXMpyRGjdmuCYVyr4i6QXBuFxKGskf015eqa3/0A6gr60mnH+rRjCy7aqO98/MC3JaLELGGqScsbuepB/t8ivoBJX/0nUWtMAS+awZsRFwSWANXIlc7g6DnFnqyhRIGqN7+oXabImtw/3VyCZApJxTCmCJprRYH5UdDi5mK3LEmOyndiGa2JOeJmD0XUv3jZTVTsVQoOaOJdjr1zCNGoOkXl1CJ7ia/buBHw==
X-Microsoft-Antispam-PRVS: <MWHPR03MB32803855063D9056D873F50289D20@MWHPR03MB3280.namprd03.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(133145235818549)(236129657087228)(148574349560750)(7411616537696)(247924648384137);
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0;
RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6095135)(601004)(2401047)(13018025)(8121501046)(5005006)(13016025)(100000703101)(100105400095)(10201501046)(93006095)(93001095)(3002001)(6055026)(6096035)(20161123556025)(20161123565025)(201703131430075)(201703131433075)(201703131441075)(201703131448075)(201703161259150)(20161123563025)(20161123561025)(20161123559100)(100000704101)(100105200095)(100000705101)(100105500095);
SRVR:MWHPR03MB3280; BCL:0; PCL:0;
RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(400006)(100000804101)(100110200095)(100000805101)(100110500095);
SRVR:MWHPR03MB3280;
X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; MWHPR03MB3280;
4:iT2FRlPDxKI37NgdDh1nqiv3ZmzQzgwZCIOOOj?=
=?Windows-1252?Q?eyXtf4GrQBMPrZ6jxzx+kGI813iXYVqkuuwbsGHwSPsIUt26ASPB218j?=
=?Windows-1252?Q?EFmsXCZKy3trUsOUu6dAiAUGibCs0NabGYdEfewQfpkZ5aOUtPsaROsZ?=
=?Windows-1252?Q?34hE8/r2iNCFkKPvfUHFyLDOAYZREqMbjA8BGdAqMmL93wDIfn16WFaG?=
=?Windows-1252?Q?c2z0b8Uuav9pAPdmgtKhjte1HsHjYvtpoWp6v35vnXCtXGMiGlOhur3D?=
=?Windows-1252?Q?aipteYslA+Fh1NVIBsodaN98qamCAIzTdyr/0pudZLg60XO6x2vfVMnq?=
=?Windows-1252?Q?qNNWiQTd5bP/5xLMuVpzE53UXvOWyguwMvWOuHtQ4MSIzjBDSd0UhqM3?=
=?Windows-1252?Q?pSjW2tP7iUJtozWC5I3DX1QB1KO/nXkZiXU+jZ368yWUwdX3Df8cxdGM?=
=?Windows-1252?Q?lPmrkuOLSi3eEoCAoaKgxLFehV7Yi5eCecbCk3LJXfwEzJXxZt/BTiN2?=
=?Windows-1252?Q?WhgyYcKGc0YJnW7Z5cplpgZMQeKsRZ4aHbG7WNUSWrYrfLsvu8ALEXTH?=
=?Windows-1252?Q?cuBNCBXBoYDxhLrlB5gULNPDwQN+loRjpdyCQs/8GtiP5oSF/HKXP26a?=
=?Windows-1252?Q?FEgtWzT4+GlNlI2fjpf1WyYXKAa1Ok1i71KLsEvPtnOpw4oyntvU7ila?=
=?Windows-1252?Q?a3pujNeBxNybi1qBOZxfq/0behBbX3pD+zGWTRiRujALxYEz7a30CxLS?=
=?Windows-1252?Q?3b/uhM67JVAxBRVyhhEpksYpB8EWlBjebPIIjIIqQsUjwfkV4JsFY6AT?=
=?Windows-1252?Q?Gi1mzM+f4b6umit8J4C9zUFWPycDDAdIFaFgKTgU8rePhWujI/FCFCdT?=
=?Windows-1252?Q?GBD1D7mNUNtcG08uAzLE5lGO2PdoJjVbYlUHauapdYqKlY4rWDp5ZSHy?=
=?Windows-1252?Q?RlAOxG/TAzsWEP+2JhGCulKtOU1FUl4ITuKbAgcJEvHMeDXiuHmTg6EU?=
=?Windows-1252?Q?EB0dE9w1v84ZE15cZWWoKJoaJCBeBG2YiQimvE/dLxKBSrQvBDkV90+8?=
=?Windows-1252?Q?27TNPGHs+kiufXPkn9XWTVjEtSOwxJksYvYqQ29vFLfPQC8E5aib4s5t?=
=?Windows-1252?Q?w9Chp4VT1Glb9nnq8eNV9DpTqsUbkF74unfZO+yKVoGWgZn4GBwTq5ju?=
=?Windows-1252?Q?EPh6yGoTa9cEfMc3D5xUDbWfTK2FwahYdhpmYVeLl8QFQPxbZ5umC5dY?=
=?Windows-1252?Q?skek78R818rj18u+Yi5RTrGemDwyvwgGyKfqwJUoYqJdeLWOwUtelG31?=
=?Windows-1252?Q?KuKGT/5ZaYOxnYQmLSNJYltf+OLM+VZKxtoVvVdGa1bT9DH6KICsh69Y?=
=?Windows-1252?Q?9kax4yld1JS1jPivdK8aHy2v7d3CLZaXbqTmfPcAFIfPL/7TOfYKZ4KE?=
=?Windows-1252?Q?zkU1XX1eV15KjrUA27UJ+42cVgrhpRPE1BdX0ccfRpOmfHStRFMjLvkx?=
=?Windows-1252?Q?w/OnOCSdEw7fJ/yKmLvwuFawp6?=
X-Forefront-PRVS: 0353563E2B
X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; MWHPR03MB3280;
23:+zAoJS9U3Gqzd9pm994cyEUh7RDOJImd/1F0U?=
=?Windows-1252?Q?trkSf1RU9zus3vGjX9CzkEjmuOiuitW9VQd0BhUmpI4VX5obfOrLPaWU?=
=?Windows-1252?Q?d/sVH255dpDFWyPFqNC2lrPopywqr0sT7/dU3dNzr0A+97FmTNbQFU2t?=
=?Windows-1252?Q?DmP9usSL6p9X6hUS0Q1p55jqAr8eZh3rf8ojeLZ/cAI22/mpXuFvnYuG?=
=?Windows-1252?Q?4K8bHCtjqTPkgXPY7G3IB9yWCi3DJfT1kWxxKIxfymwNpYdy7guiHdQm?=
=?Windows-1252?Q?h9ZQjHBwijUxULcrmTjqEc1pP2894pnsB4UoUGDcXsf6uEBlLHdBN0WO?=
=?Windows-1252?Q?DkzCTu5hDDa/GvHN8bj7dgF7mHHsvaUu6XbeLjnjo4P1+MtHh97R4FpW?=
=?Windows-1252?Q?BUAb4iCGgCR3Zs9t0wbDr+cckI+aDanDs9glyvNcH1HuxKaTnfQX4Fce?=
=?Windows-1252?Q?EJV45d2oQlHRWaYdkz+LbzMLdYQcvAHHqTYGwmU7gel4fHKmBESoeZoW?=
=?Windows-1252?Q?/8apWQed0dzgMRDqbxIZhrTGuMePRJh8BvjCy+Qhzn+ErDc7mVIKseIJ?=
=?Windows-1252?Q?vqABkbaFqV7uAa+kT3ErqRqdvFESzv/jo8HJYLFr9PjQ0XMS0oc1qXp/?=
=?Windows-1252?Q?pvEdfBdcN6KDsrKss4RTruspSSJhZu4Lx6x6IEd8rurZbtZNSkrLIs9w?=
=?Windows-1252?Q?keO2+Jc2QPX6ZwcaLh64lEyZiSzb28xXKnbtT7UcP4DHWkNz4d3PFAwb?=
=?Windows-1252?Q?4cM5h2ZSi6vtKphy50qSq3zgqbHQW2YbOB+CE0qTXR9MZO2ZoI/kvKjr?=
=?Windows-1252?Q?obNhuZEzYYG44eb8lB12ptlavIThKgGjvIB828ekvsnsMMWDpIzHBnvV?=
=?Windows-1252?Q?CVow6xprK8cWKKyZwVfRtud/DgoIVCaZA57rmUvFDMog9+rQB56eVfoH?=
=?Windows-1252?Q?VCR9MysFxT4icIV4sCnhQEdR0sfoSbvfGyk9NTFCVgawcAkXpvWErW3L?=
=?Windows-1252?Q?SgjLk0IP2mVnPuYGH3Y6g1Kw+TfKtC3DOe7K3KkDozHgWsSAoMfi2JDV?=
=?Windows-1252?Q?gBy9TcajXEDheXmnHdDyR0pAqO3e9edVX6wGOStpMFQ/22FIX36FM+Ub?=
=?Windows-1252?Q?VvdQrMfE2UtFiZ92KingaNI3LJJ8FvghMo4mQNiB2cKvsZvL3at591Kl?=
=?Windows-1252?Q?g65p0T/Ia8eMseD9los5ZydXD6W8x6Xiuf9v+Tk4l5My8PtXEuQPQeh/?=
=?Windows-1252?Q?MfQT1s4eYRvkoZUW8Z928GhpG3dmUVyj+P4gprgTsDtmsxeSn4IJ+3Y5?=
=?Windows-1252?Q?XEeFOffqJ1vwv5KFi0Kz5ahFE2ngjMjKvjfaoVQ17tdlZsjYpEPswkpC?=
=?Windows-1252?Q?wVgz4zga7oQ6eMK+v0y5LSviwOypskEKH89E/cDjG9kQ0yAtK+7NTsG6?=
=?Windows-1252?Q?Rx8HVQDaHBG+AG5CpRplFQQ0fyLeDwGZ71Jby9JEGgLlKaC3XEivHlBl?=
=?Windows-1252?Q?/WxBSAyUN9NQpvB/ZC/ELAFePeP8ZSXOKiRreKU9XETnktwMR0y+3cfn?=
=?Windows-1252?Q?t/I1/hRZFcbgqI058s4vYo6fJG00zxTd+MtYk2RiSfvJ6HYWssnT0rBd?=
=?Windows-1252?Q?UETu/5eGm1vhDhMsJeoVci+TAqOrYu12RhPFf5otNTLAMptxdr9l7g/e?=
=?Windows-1252?Q?JkfELzz8w=3D=3D?=
X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; MWHPR03MB3280;
6:frNiX7bTtcrsZqQsDwTxqjzfY3jySvkuxRdgwd?=
=?Windows-1252?Q?xekp2TDzTaP2vAACybmW58AirJCBnp9KI/IGlO2wOuLOoH28Z0POou6M?=
=?Windows-1252?Q?S+52PdZ5rL4mNOWjo2o38pGyEC781BxW/4RWTxeIYzUi+SZHtb8I7E6V?=
=?Windows-1252?Q?3XQD7dICSqywEJzrnvyDtKNxt5OOgZfoWYVOa5+Lakmq/ljZSzpwceLo?=
=?Windows-1252?Q?1bb3oWmXgBSRuboTugpS52dqZfi03BS59xdJFZKB1EDyzy9Kadh+ATqC?=
=?Windows-1252?Q?g4p6eGt4Ce3eHbnmuWaqfjbsoIPtXzwI+XWU1Vxs4NDa3zoEWNHkhNpi?=
=?Windows-1252?Q?l0OsLSQKqrLQDUEBF6goScxbY++XDNBGylZ9St46p9qk5YXNaHWLiIXH?=
=?Windows-1252?Q?0FtquzZXVlmubzBZ34IQrHlyEZDrYC4z4MYThfwXnGR1mZyuOysjTcsI?=
=?Windows-1252?Q?PAHbRDDy3wwfXys4w78+V7V67hl9NgGVi2tvQs6B3hioUf2VHDxpBQ5l?=
=?Windows-1252?Q?EpqB7IimmTVFzS+EwoXUl2yVFIURyf8Mfp4lfwIYZi6cwTIEaTDo0xV/?=
=?Windows-1252?Q?x4YnzCDIYn0lPoA9DrTEtENUFDaQbCtCSSViQd2SXGMmNJH5LGkyQudL?=
=?Windows-1252?Q?LD/nI6LgC8eXWF0+5zCYZ2ibWebMtuFhuGnzdGqpw9FytNJnlEMyaaya?=
=?Windows-1252?Q?ObanfvGoSMuzyZAtE4SIJO1GlOB/q1uDW0c6sFDhYD0ozgXEvDggbO4E?=
=?Windows-1252?Q?C36xlgw6+1ChwYRGjqEcq+SUWRnqfRruj/O8/Ynaloxl/XixZvSH4Csd?=
=?Windows-1252?Q?GvbFWm3c9RiEOUPXuN7DuChbq7kPg5K/91+od2p20rlTXd6DOM6kNcPx?=
=?Windows-1252?Q?UgeXjLNI2ch0VThp8bRJxtaFjk6T8K496zYnTwzDPdPiOKjHJ1c61sj5?=
=?Windows-1252?Q?1PhWTYWCTVE6hn0qSM8siszTrJiy/gAHeTbX9hK/oTFdc4vMH7kvQOcA?=
=?Windows-1252?Q?M5hdiV0G7laRPxjQweWyk56T2SHCpuIltBnEUx4JpKQhh+d57lnMD91F?=
=?Windows-1252?Q?NiZfVUBozYGnQ=3D?=
X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3280;
5:rJpGqIdc91LPG2PDzQuDyYkkxo1gdINswBP9AVQG51RpmwsDuPQZGUH1+wg1IFUJcGUWruaaHngNUJZIbOZDJMvOaafszehEuFyrOCb2x0a2zw+uOuGYGYB2aplNSj+O+Yy+Ct/FSyLiLbEvAgRsZ7yTmHyzkQVe1fUs12aMmk8k/J9hji5D9NC0R2HuR6pfXPzA+IxbF60xXUbPRbzwHkMWGPYH4g/NNJsHPg/j7GoE22ryxvYODYOoVn+AWr5VCtn+gMVYT0Rgz8RzU07O8RZ3apcBuHcg8E7+WGZFyVhszzdy9ZF2sCKTr59kA7eQiQ/Y74RvmddmJpy9tTPWmo25BNei8C2PBXDWWyipRCJM5F0fW6ax7G+klK2+B1nUU2N98LUPtDCNPa+SGww+4ueAL38vbMuT0dxcZ5a7q1oUDTPy3S9/gsrO/san8uTkJisWs9u5Nt1rGQ4G/WFgYraYVJhSMPD3JOTIGDXWo+52bqzJSHlrGUApqZHF7RfLXmwF1aOjwMypHqVjo72RFw==;
24:vXx2Isavc/TyTbXkK+Ta1NBusm+SIylBNTeWfI9q4d4e249VMLPEpjJaXVN2Fj2r+wlmho4vCfMkBpIphRvCviT3LwX+Y9SKT0zgLUZRPik=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB3280;
7:LvDLzj3RVyV5Bf7b62py6sXkGPr17LgEBsVZtuULYfUPvdDa5gJo/KjzyI3RoN9Axfp09kcKcRC2xPlPeFjsizy1p6SV1Da2Nrs/KYe34+oIQPF3qXGAs4DW6DiEGM4o+djdcclwZohpIPDy8uV8U97Ae20Gi4FUWZTcxG5epSC6n/FVgtWPBba22vRgSDDhpfbydBRzJ3jXGozF8IVzJsbnHSu7TqBqRSNiD2bnGDS3UIfPilVu2kiQxlLOUpcHAIBtsQTD2TBsb/1R8+BGUI8ZS6tujaAy7cIYwTG/rO8GBQ5Mz+f16Lq9TXWL4V28Pw9vYs9BEbs8KV5KxJPmO7mTJ8QC9MtqOhz2HVqUwwnD07Lm+s59nIWfgp4UOY5HCAGRWtl64ewZGBCHFaE+65vs9n1ZRmWeBCy5aC5eMxIfK6SH9omU02rJkF/anEdlpMJ0Vh04XGVbTfAcTR6BN9E2aCfb2hQU63KLQa1Yf+ykp/cH3+wuy7vRP/GEu7Ka0NfJd/EwVL6kwbOI+2Zflqf6yVirBWXir6thjK2hPYRP0hxZ1+vyiXZl0smqPWiNieJx0GSgGRkLCSfng5N+azNWiwrpkXhae4jdiu/afxIPjTSKCUhVGD09FnuUwknGvwix+41fhggaN+rfO8zujdqh2SFLoTyzx+hgVDxjDn8hK3aEmUXTOCXQ/3dtFTvubTkrF/wXlqpJqK+8c0EmZyFTaPWYMsKNjSwJf0eS1NO9XQNxvFCy4OHUP859/mzjYZBJnikuAx4gqBUI++UNjcvIbcMOF7RIbRM3Bl66l1s=
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jun 2017 05:32:32.4002 (UTC)
X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50];
Helo=[tx30smr01.am.freescale.net]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB3280
Subject: Re: [dpdk-dev] [PATCH v9 1/2] mem: balanced allocation of hugepages
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Jun 2017 05:32:38 -0000
On 6/27/2017 3:54 PM, Ilya Maximets wrote:
> Currently EAL allocates hugepages one by one not paying attention
> from which NUMA node allocation was done.
>
> Such behaviour leads to allocation failure if number of available
> hugepages for application limited by cgroups or hugetlbfs and
> memory requested not only from the first socket.
>
> Example:
> # 90 x 1GB hugepages availavle in a system
>
> cgcreate -g hugetlb:/test
> # Limit to 32GB of hugepages
> cgset -r hugetlb.1GB.limit_in_bytes=34359738368 test
> # Request 4GB from each of 2 sockets
> cgexec -g hugetlb:test testpmd --socket-mem=4096,4096 ...
>
> EAL: SIGBUS: Cannot mmap more hugepages of size 1024 MB
> EAL: 32 not 90 hugepages of size 1024 MB allocated
> EAL: Not enough memory available on socket 1!
> Requested: 4096MB, available: 0MB
> PANIC in rte_eal_init():
> Cannot init memory
>
> This happens beacause all allocated pages are
> on socket 0.
>
> Fix this issue by setting mempolicy MPOL_PREFERRED for each hugepage
> to one of requested nodes using following schema:
>
> 1) Allocate essential hugepages:
> 1.1) Allocate as many hugepages from numa N to
> only fit requested memory for this numa.
> 1.2) repeat 1.1 for all numa nodes.
> 2) Try to map all remaining free hugepages in a round-robin
> fashion.
> 3) Sort pages and choose the most suitable.
>
> In this case all essential memory will be allocated and all remaining
> pages will be fairly distributed between all requested nodes.
>
> New config option RTE_EAL_NUMA_AWARE_HUGEPAGES introduced and
> enabled by default for linuxapp except armv7 and dpaa2.
> Enabling of this option adds libnuma as a dependency for EAL.
>
> Fixes: 77988fc08dc5 ("mem: fix allocating all free hugepages")
>
> Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
> ---
> config/common_base | 1 +
> config/common_linuxapp | 1 +
> config/defconfig_arm-armv7a-linuxapp-gcc | 3 +
> config/defconfig_arm64-dpaa2-linuxapp-gcc | 3 +
> lib/librte_eal/linuxapp/eal/Makefile | 3 +
> lib/librte_eal/linuxapp/eal/eal_memory.c | 120 ++++++++++++++++++++++++++++--
> mk/rte.app.mk | 3 +
> 7 files changed, 126 insertions(+), 8 deletions(-)
>
> diff --git a/config/common_base b/config/common_base
> index f6aafd1..660588a 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -103,6 +103,7 @@ CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n
> CONFIG_RTE_EAL_IGB_UIO=n
> CONFIG_RTE_EAL_VFIO=n
> CONFIG_RTE_MALLOC_DEBUG=n
> +CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
>
> #
> # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.
> diff --git a/config/common_linuxapp b/config/common_linuxapp
> index b3cf41b..64bef87 100644
> --- a/config/common_linuxapp
> +++ b/config/common_linuxapp
> @@ -35,6 +35,7 @@
> CONFIG_RTE_EXEC_ENV="linuxapp"
> CONFIG_RTE_EXEC_ENV_LINUXAPP=y
>
> +CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> CONFIG_RTE_EAL_IGB_UIO=y
> CONFIG_RTE_EAL_VFIO=y
> CONFIG_RTE_KNI_KMOD=y
> diff --git a/config/defconfig_arm-armv7a-linuxapp-gcc b/config/defconfig_arm-armv7a-linuxapp-gcc
> index 19607eb..e06b1d4 100644
> --- a/config/defconfig_arm-armv7a-linuxapp-gcc
> +++ b/config/defconfig_arm-armv7a-linuxapp-gcc
> @@ -47,6 +47,9 @@ CONFIG_RTE_ARCH_STRICT_ALIGN=y
> CONFIG_RTE_TOOLCHAIN="gcc"
> CONFIG_RTE_TOOLCHAIN_GCC=y
>
> +# NUMA is not supported on ARM
> +CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
> +
> # ARM doesn't have support for vmware TSC map
> CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
>
> diff --git a/config/defconfig_arm64-dpaa2-linuxapp-gcc b/config/defconfig_arm64-dpaa2-linuxapp-gcc
> index 2304ab6..f78449d 100644
> --- a/config/defconfig_arm64-dpaa2-linuxapp-gcc
> +++ b/config/defconfig_arm64-dpaa2-linuxapp-gcc
> @@ -45,6 +45,9 @@ CONFIG_RTE_CACHE_LINE_SIZE=64
>
> CONFIG_RTE_PKTMBUF_HEADROOM=256
>
> +# Doesn't support NUMA
> +CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> +
DPAA2 does not support NUMA so,
CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
> #
> # Compile Support Libraries for DPAA2
> #
> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
> index 640afd0..8651e27 100644
> --- a/lib/librte_eal/linuxapp/eal/Makefile
> +++ b/lib/librte_eal/linuxapp/eal/Makefile
> @@ -50,6 +50,9 @@ LDLIBS += -ldl
> LDLIBS += -lpthread
> LDLIBS += -lgcc_s
> LDLIBS += -lrt
> +ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y)
> +LDLIBS += -lnuma
> +endif
>
> # specific to linuxapp exec-env
> SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) := eal.c
> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
> index e17c9cb..647d89c 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
> @@ -54,6 +54,10 @@
> #include <sys/time.h>
> #include <signal.h>
> #include <setjmp.h>
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> +#include <numa.h>
> +#include <numaif.h>
> +#endif
>
> #include <rte_log.h>
> #include <rte_memory.h>
> @@ -348,6 +352,14 @@ static int huge_wrap_sigsetjmp(void)
> return sigsetjmp(huge_jmpenv, 1);
> }
>
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> +/* Callback for numa library. */
> +void numa_error(char *where)
> +{
> + RTE_LOG(ERR, EAL, "%s failed: %s\n", where, strerror(errno));
> +}
> +#endif
> +
> /*
> * Mmap all hugepages of hugepage table: it first open a file in
> * hugetlbfs, then mmap() hugepage_sz data in it. If orig is set, the
> @@ -356,18 +368,78 @@ static int huge_wrap_sigsetjmp(void)
> * map continguous physical blocks in contiguous virtual blocks.
> */
> static unsigned
> -map_all_hugepages(struct hugepage_file *hugepg_tbl,
> - struct hugepage_info *hpi, int orig)
> +map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
> + uint64_t *essential_memory __rte_unused, int orig)
> {
> int fd;
> unsigned i;
> void *virtaddr;
> void *vma_addr = NULL;
> size_t vma_len = 0;
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> + int node_id = -1;
> + int essential_prev = 0;
> + int oldpolicy;
> + struct bitmask *oldmask = numa_allocate_nodemask();
> + bool have_numa = true;
> + unsigned long maxnode = 0;
> +
> + /* Check if kernel supports NUMA. */
> + if (numa_available() != 0) {
> + RTE_LOG(DEBUG, EAL, "NUMA is not supported.\n");
> + have_numa = false;
> + }
> +
> + if (orig && have_numa) {
> + RTE_LOG(DEBUG, EAL, "Trying to obtain current memory policy.\n");
> + if (get_mempolicy(&oldpolicy, oldmask->maskp,
> + oldmask->size + 1, 0, 0) < 0) {
> + RTE_LOG(ERR, EAL,
> + "Failed to get current mempolicy: %s. "
> + "Assuming MPOL_DEFAULT.\n", strerror(errno));
> + oldpolicy = MPOL_DEFAULT;
> + }
> + for (i = 0; i < RTE_MAX_NUMA_NODES; i++)
> + if (internal_config.socket_mem[i])
> + maxnode = i + 1;
> + }
> +#endif
>
> for (i = 0; i < hpi->num_pages[0]; i++) {
> uint64_t hugepage_sz = hpi->hugepage_sz;
>
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> + if (maxnode) {
> + unsigned int j;
> +
> + for (j = 0; j < maxnode; j++)
> + if (essential_memory[j])
> + break;
> +
> + if (j == maxnode) {
> + node_id = (node_id + 1) % maxnode;
> + while (!internal_config.socket_mem[node_id]) {
> + node_id++;
> + node_id %= maxnode;
> + }
> + essential_prev = 0;
> + } else {
> + node_id = j;
> + essential_prev = essential_memory[j];
> +
> + if (essential_memory[j] < hugepage_sz)
> + essential_memory[j] = 0;
> + else
> + essential_memory[j] -= hugepage_sz;
> + }
> +
> + RTE_LOG(DEBUG, EAL,
> + "Setting policy MPOL_PREFERRED for socket %d\n",
> + node_id);
> + numa_set_preferred(node_id);
> + }
> +#endif
> +
> if (orig) {
> hugepg_tbl[i].file_id = i;
> hugepg_tbl[i].size = hugepage_sz;
> @@ -422,7 +494,7 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> if (fd < 0) {
> RTE_LOG(DEBUG, EAL, "%s(): open failed: %s\n", __func__,
> strerror(errno));
> - return i;
> + goto out;
> }
>
> /* map the segment, and populate page tables,
> @@ -433,7 +505,7 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> RTE_LOG(DEBUG, EAL, "%s(): mmap failed: %s\n", __func__,
> strerror(errno));
> close(fd);
> - return i;
> + goto out;
> }
>
> if (orig) {
> @@ -458,7 +530,12 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> munmap(virtaddr, hugepage_sz);
> close(fd);
> unlink(hugepg_tbl[i].filepath);
> - return i;
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> + if (maxnode)
> + essential_memory[node_id] =
> + essential_prev;
> +#endif
> + goto out;
> }
> *(int *)virtaddr = 0;
> }
> @@ -469,7 +546,7 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> RTE_LOG(DEBUG, EAL, "%s(): Locking file failed:%s \n",
> __func__, strerror(errno));
> close(fd);
> - return i;
> + goto out;
> }
>
> close(fd);
> @@ -478,6 +555,22 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> vma_len -= hugepage_sz;
> }
>
> +out:
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> + if (maxnode) {
> + RTE_LOG(DEBUG, EAL,
> + "Restoring previous memory policy: %d\n", oldpolicy);
> + if (oldpolicy == MPOL_DEFAULT) {
> + numa_set_localalloc();
> + } else if (set_mempolicy(oldpolicy, oldmask->maskp,
> + oldmask->size + 1) < 0) {
> + RTE_LOG(ERR, EAL, "Failed to restore mempolicy: %s\n",
> + strerror(errno));
> + numa_set_localalloc();
> + }
> + }
> + numa_free_cpumask(oldmask);
> +#endif
> return i;
> }
>
> @@ -562,6 +655,11 @@ find_numasocket(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
> if (hugepg_tbl[i].orig_va == va) {
> hugepg_tbl[i].socket_id = socket_id;
> hp_count++;
> +#ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES
> + RTE_LOG(DEBUG, EAL,
> + "Hugepage %s is on socket %d\n",
> + hugepg_tbl[i].filepath, socket_id);
> +#endif
> }
> }
> }
> @@ -1000,6 +1098,11 @@ rte_eal_hugepage_init(void)
>
> huge_register_sigbus();
>
> + /* make a copy of socket_mem, needed for balanced allocation. */
> + for (i = 0; i < RTE_MAX_NUMA_NODES; i++)
> + memory[i] = internal_config.socket_mem[i];
> +
> +
> /* map all hugepages and sort them */
> for (i = 0; i < (int)internal_config.num_hugepage_sizes; i ++){
> unsigned pages_old, pages_new;
> @@ -1017,7 +1120,8 @@ rte_eal_hugepage_init(void)
>
> /* map all hugepages available */
> pages_old = hpi->num_pages[0];
> - pages_new = map_all_hugepages(&tmp_hp[hp_offset], hpi, 1);
> + pages_new = map_all_hugepages(&tmp_hp[hp_offset], hpi,
> + memory, 1);
> if (pages_new < pages_old) {
> RTE_LOG(DEBUG, EAL,
> "%d not %d hugepages of size %u MB allocated\n",
> @@ -1060,7 +1164,7 @@ rte_eal_hugepage_init(void)
> sizeof(struct hugepage_file), cmp_physaddr);
>
> /* remap all hugepages */
> - if (map_all_hugepages(&tmp_hp[hp_offset], hpi, 0) !=
> + if (map_all_hugepages(&tmp_hp[hp_offset], hpi, NULL, 0) !=
> hpi->num_pages[0]) {
> RTE_LOG(ERR, EAL, "Failed to remap %u MB pages\n",
> (unsigned)(hpi->hugepage_sz / 0x100000));
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> index bcaf1b3..4fe22d1 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -186,6 +186,9 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
> # The static libraries do not know their dependencies.
> # So linking with static library requires explicit dependencies.
> _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrt
> +ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP)$(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),yy)
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lnuma
> +endif
> _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
> _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
> _LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
>