From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00046.outbound.protection.outlook.com [40.107.0.46]) by dpdk.org (Postfix) with ESMTP id F3D661B1F5 for ; Thu, 11 Oct 2018 12:02:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gpTggYIqL5yO1Xdcv9/PsLmcqNPdXpwHXyXHfradK/Q=; b=bdY9fG44h6KrEKbsM6l7sqglycXd7LNM0DVNi+P9SfrArDzexJMrruwIIzKgGEZSJNtZ2Bb1tNCYPBu/SQe0c8ezH/K9ozUby2ce+/+8BNPjCYzi6anoYWppHOasHxy2SElWkns5qdQbupAdzU4AXdQb6kdopTvFGHWfVJdjFEM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; Received: from [10.232.14.39] (14.142.187.166) by AM0PR04MB4676.eurprd04.prod.outlook.com (2603:10a6:208:75::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.26; Thu, 11 Oct 2018 10:02:54 +0000 To: "Burakov, Anatoly" Cc: ferruh.yigit@intel.com, dev@dpdk.org References: <20180925125423.7505-1-shreyansh.jain@nxp.com> <20180925125423.7505-4-shreyansh.jain@nxp.com> <894130a9-017c-348d-31f8-c4c23f517f25@nxp.com> <2d2e9008-fb5b-3ecd-2d2c-e86250f5d363@nxp.com> <5e153c76-6eaa-e6a5-28ed-7cec191d4581@intel.com> From: Shreyansh Jain Message-ID: <2c377b57-c418-5c03-a23d-5da91ef898d0@nxp.com> Date: Thu, 11 Oct 2018 15:32:24 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <5e153c76-6eaa-e6a5-28ed-7cec191d4581@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: BM1PR0101CA0028.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::14) To AM0PR04MB4676.eurprd04.prod.outlook.com (2603:10a6:208:75::18) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3722c8de-99ef-4881-7910-08d62f60b6d6 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR04MB4676; X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4676; 3:wGo14Va2gCuAvBK7K4bOu5kRoIhU4zPXoulFy4zPeWV+uhLVV8iHTsAQI1hL9BrhTcuftLGrR/iRmESXdJyYs1NVV2Pm1SMdl7uZcWCQdcA6Zbgdu7KeddvkxrkMvDlNa2gMhTuTzuDAcz6gjVoDZyQIoNRjfOKiu4qRsd1KWMXKmp3r+/6aifh4PYsL8ekVPBn4woACqz42/JrVFVBGFYjYsjoftObxG1y7gQ7VIDPX3g+w7IvOFAvOpH600JSS; 25:QgT9ArDm0Ms5O/Lf1naf/gINcH/n30o83Z56xJnTcm7D36+mHRC3K/heplZY13QDhwYn/wW8hGDkGyHQNJBawwyDflFfmOUpqwtpc98Lus0lp/iyXFChfIXxkrfPUA6ozZDbFh57OMbdew1BZDYWsGIm27a4/KqK0IIszWrCUoitE+Ai4NpLMRdIIhzfsg2Le1RewpsuHUPMz7Pm/7qD5WYY6Z/exRTEJ3l+QUJPk1ZlcyCeynXSg20VJofee3jV4q/3htYNk7qGX3tQ1KxsTQXv3yICRdbFy70Fgca6xksdedrwxZy+oFx4kLtm3axzJNR59i/+U3kqK3thS3ZZjw==; 31:dCQc3B0exY4Ohb5JH7yBZPWf9V3/wqLYZr2lECoNCbxkQtgkEvM4RBOjJYQnxCDUynnWB85QhEoD2boiGGZCt1iPyhJEHh3zbDZBbj7WtX+wNhOHpDj9XzAe1MCgT/CHfMYUwBoPggG2lPzTsXqCiG50fSU6rBievrk32gLM6qutBggmRO/qfPfAn32oJb7FRwJ6MHJ+vqp/pj4xPty2CDoEw0I+A/rSdlVGuNx2j6M= X-MS-TrafficTypeDiagnostic: AM0PR04MB4676: X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4676; 20:axMnY6R5kpdbd9Ax+2XKhSsI4ScJof+HWRQXhSsPMqikA7B1+PKUj1zmXsQ5+34mM3LdbMDZIf2oGoVBnnfUp9CN098bWgkvq5DeD8fXYTBYUx8/yL6JoDAJxsZuZOkMXmOhe4BVs+wPkTE6nrc7lSn5uOf9CIGyMdICcb5+80UnZtalzKi2SJN0AhBYIVwxdXXVphtXzPr7X8okjWFu9HblOYNLF/Pd5bKEd03a0VqbiS5X1AAnBESgtso6bDsKx7uchBl/23hU1shvumezYEsSeH7BCwiKiotL6Q82ZZhmUI5ohmHxeZq3Oo7EUWrwGoCIJENZQaJrJFAQzhvnigJ3+3IsUIrtbN0iVBR084sgQMglpWf5owvvWVNficTBqH5KwfrxQnrP9kweO7i7oq5fbHmIqogcdALUTOnVsHZOmZ0bnaKDOEH2c9ALmBdbNuL5DbI35ZtJ99pGsLc3BYp6+QOF/ddYddU60afprqmRoTE9a6EoukCU/3cJGN5w; 4:bSYuqbpug2OF7tm0gHeosLb9DwlogoYBNp+PMSLxS6wiIuUzYPcTA0ABmUrLbjxMC4+cq70e6dy9scIhLxihrxAbhdLjuI6Mz0vQknhcM8a75Yq4B1ZGiFmBzk8hib/BhzLuq9oZrIx78uPlppo0XEoYOwkadorj2ZpoJau0coJQXQdychWCIyA5isuDgjEPyim05WRZWdAyiyBHqDM5lKuqu14TzECeEwJpXxfM538o4oHkFF2H96zj16mieupfazHDenW6iSql4CtWgek/9/bzxS9zQrq9uNhCs8Mt8K4UWIT+0wYmZNpgk6l8qOzI X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(163750095850); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231355)(944501410)(52105095)(3002001)(10201501046)(93006095)(93001095)(6055026)(149066)(150057)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123562045)(201708071742011)(7699051); SRVR:AM0PR04MB4676; BCL:0; PCL:0; RULEID:; SRVR:AM0PR04MB4676; X-Forefront-PRVS: 08220FA8D6 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(136003)(376002)(396003)(39860400002)(366004)(346002)(189003)(199004)(105586002)(106356001)(8936002)(14444005)(67846002)(53936002)(64126003)(77096007)(26005)(186003)(16526019)(68736007)(1006002)(31686004)(81156014)(6486002)(6246003)(6916009)(5660300001)(8676002)(229853002)(93886005)(81166006)(65826007)(25786009)(76176011)(50466002)(386003)(23676004)(2486003)(52146003)(52116002)(97736004)(4326008)(53546011)(55236004)(2870700001)(86362001)(486006)(11346002)(476003)(956004)(2616005)(31696002)(44832011)(47776003)(316002)(7736002)(16576012)(6666004)(2906002)(478600001)(446003)(305945005)(65806001)(58126008)(36756003)(3846002)(65956001)(6116002)(66066001)(110426005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR04MB4676; H:[10.232.14.39]; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTBQUjA0TUI0Njc2OzIzOjg1aWtrSFRVN0ZuNllZc2d3dnJNQXVaUWxU?= =?utf-8?B?MXhscHhZWDhsaGthZGNyL2JLYTVEWE1QbG1wV3NWc0JWOXhHSmxLZ241VVpx?= =?utf-8?B?RDBON1FuZEJXOW5xQjhoc3I0Z1Zad0VVT3pYQ0Nhc05VbitCNHUrOUp2Sy9P?= =?utf-8?B?THFuVmVqMXF4U0ZvbjFGU0gwRHhJcDk4VGxQa3RxS2w0UWg2OXlOYzlYOVF4?= =?utf-8?B?citsT0UwNGtoTm0vUjNMclhOS2srNVpmMFRJQ1p2Wm5wUFF3V21QWGNwSGNk?= =?utf-8?B?RlUyVUNNRGtMc1pHbWhCTmxtdmxTOWJJTnpMNEljYXNHS3pZb1VEL0dQeERt?= =?utf-8?B?R1JERFNDYVcxdmdYTFpLSU4rOGIzUnZYcVFsVlFqR25zNHlyeU92VUNJc3p5?= =?utf-8?B?bjNEV1RFck05bHFyZEowdDNSVlprOXVTS09ZcXhXMExsUm4wVnhFQmI5Z0py?= =?utf-8?B?QnpOTEJ5djNseDZOcVZGcWdDSGp2cGZZaXRlOXpOdXAzMWYxMjlIeDVuQ202?= =?utf-8?B?czMvMk9GRnRRL0N2NDVHUnZ3bDQ2bnlDbk4wM2dpclhFbFExbGZKYTZHekNh?= =?utf-8?B?aTRLaDNlMjRFbzRjZUVtZE1RQUFobWVjNlI0bGJJTGYvbDZQekkwWGRCMjYx?= =?utf-8?B?N2RSZ0svZzE1dlBaVkNhN3RRWUlJRUdXTVJUSXBwUHcyZ0pPYWRQTm5IYTRz?= =?utf-8?B?cDZsNkpFSXNxbU40WXRBNnlSVkc4dVRKZWpDWGpBMXFRNzhYblNwYlNxVG82?= =?utf-8?B?ZE0zM2p6U0VoOFlxWnRaL3lteFpzTTdURVdBYXpOL1BuL0xYQXNGaWhFRDZ1?= =?utf-8?B?dmx1ZHROSlJQVC9wN3hYYTgzTHNWM1NpWGZCbDhFZGJhbXl0ZHU2STdaS3B2?= =?utf-8?B?SWc1aGNONjBpVnM4ZlNPK2FpaENBNkMrUVRsMWVPYWlzSFQ0dTZHN3dZcmU3?= =?utf-8?B?empxMmV6ejdNVWdLdzhUNkM5eXZzRzVyR3lSbkJNbUdrSXlRekNiaTlDV0xY?= =?utf-8?B?aFFEQW81ZVA0RmQweGtCbTFnYi9MVDFrSTNNMmZrczVNYkNnREZFTkdoS2x2?= =?utf-8?B?RXNWT2g0aHRMZjNPVFJaNnlTR1hOZTNhRkN1UmoySEZIUEN3K3g4MFRGdkFX?= =?utf-8?B?akVrQXNOQ2tTeDU2QjY1N2hhWTJ3ZUVTY1cxTzdiM1Y3Y29qNm5pTHEvN2NC?= =?utf-8?B?Wjg3VHY0cDRBT2dkWHZtUTRwR1NTS2NJcmJ2TkRsVWxYUFRXWWhuMGdocUwr?= =?utf-8?B?a1hVRERXditIM2R3b0tEeDd2Y3Q2dVVwTENPaG5SMzNaLy9tS0hiK05OTDEx?= =?utf-8?B?anNWNUNkTDJORE9kUXd5eE5NVnpSNU9mNm8vSXBkMWJsU0l4NGRNdlJ1eEdm?= =?utf-8?B?QVZadFVDdmgraEJnRXBtUmFHT29LTkdVWXliVE9CdG4vN3NxckY0NXBUUmVj?= =?utf-8?B?WmVZZkRGS2pVbGh6bFVXY0s3aUErTEVqMG9RSE0xWjFNZkZaQ0hpaXNkbElX?= =?utf-8?B?MFpJZEZpWHpLdnovc25ZZG56VVRMT3I4MDRKM2RVbzc1WFZJUkppaG5qdkYw?= =?utf-8?B?cE1rd2tBeU9BTHI5V0FzeTRWb0d1RHZmSTIvQmt1YmRaUmdQdGNsQi95QWxw?= =?utf-8?B?WnY5bTdiTDVGRUlYSjB2bFQwQkY0VEZvVldORHZtZkNzOUFvUzAwR3QyTWE1?= =?utf-8?B?aGl1eEZwdXpwOFRORnZralUzVTlZY0EyM0FFNHE5d2lrRFJNUGZQTWlUL1dW?= =?utf-8?B?YVRTR1J5MDd3cFlKanRxMTlncE5GR001ckFoR2JaQmJqeXNud3ZRWDJBM3Y2?= =?utf-8?B?cGw0WHZRNGZHY1JaRk82d3lBWFNRT25rRzVuYnNySUEwZGd1dGJrWnNZQzdO?= =?utf-8?B?MEp1blhpSjhkanZUeVZrQXA4QlA5cW9qRmtHZnVnNlVWVHhsYm8wSFMxeGFW?= =?utf-8?B?V2lwaGEzcDVFNzlra1lsVHRPZG9TWjVVTW8wN1VsVjRTRzBhZUZPbnBjdjRF?= =?utf-8?B?OUlaYXNUdXNHOEJPWmJNeGpCRnJ2cVljcDRFV0psSjNwc2ZSSVJwSWFKSm1m?= =?utf-8?B?U3kzbEtBdEUvWlFLeFpWdE53cUFsQ1YwN0czazdGdE1oSHQ2Ykc2ckRoUXdJ?= =?utf-8?B?alE9PQ==?= X-Microsoft-Antispam-Message-Info: HVRjHMAr9t10+m3chEHLglqLVg4wigL81Rn4ftxiEIkTcLZZG1HGgPW+ZvkDvXC8BljZsjyKcko+EtcSMI0hL/k8UV3tWu4kLVQgPwnCmHk7mxfiZlwsY6Nf5MCorpOlug8jsxV7UXh7DwS4w643G6dowE+MfyvSAzbkYvUGHclKaHVb/NhKA4osrKkWRI2FmjyBz3/6N2r7qQd+GGlezhLttU0fsheAZIFZjOSudL6PcficBua6Y0AfsnpyeBzztPikXj6nBjFuAjRclzAg4I79X4BLtuCflK1anRTzr1SVAFMaeBX3VRNxcYYLQY9AzYhRmI0D86qKlLEkRVJjvEuf7ygDTsCVhO353TZbEec= X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4676; 6:2TjoxacrhcY1vy68r8ZL7BzHOMI6vaSjolbDvk5Gy6STxu8G2VUlyNIHeLxQf+gZsCcrePtwWhyNiogKGyyHzlIZsckuja6hJ2cY6kMXju5PnRtB0TKurGh7yQ1Lmm6ModdqGxCKdquK2rLOIzXNfw5PCJ+h/+W6AvPAeYXYLCyxMLM1zrUVZZyoEkJf8m2VEfj0IbLLVUNCchlsO19gZ+nq4AmtyMlBjjd1NP1wY294ectobs3IRQAfRQLa8dmo6EKyykcKOgrWgrCVN1SnKUFkI76MP662O2C2V2wx8flUyx/lmhZraFhyBOz1NP3sNjQkx7KOHYr55kQuMmEgBKmY3DH+im+FLCwj40q51YxdvEzwDxmjNe0icPBh1wBdyiQJuCmC6k4k1jEttGNLDctjYwqnAFmsP5EdJg7sdGd+zT5n8eIfJQpPdxtgn/6Dp+LtD7MAo9f9T/NrSf/2Eg==; 5:qnJdXoLyIdzb+/h10yiE/FrWG414/Bm0GlpI2wKRkAFfC8PvUFqQnA3oYZtrIlaxJjyIivn/bN3O5MrdNlH4xjMV82EaNvdbQCgQR83HqvcFo2nDqQc9UbMCL5CIWwcAAK5IDhXRESUIebsiETHVx5JQA0ySS4S20NGklX1c9rw=; 7:mbQpb5vubvvVjXml75iPh9KZ5Pkn2jQlnlxdqdRY1XDEemxUpS3UaTHIQJBTuhZWWlXNc+aRcAYk78FiXCb1gj+VQ4vbikEcQB/noqTlYv186kLQRDGUH252HUvaKAWUQz27pU/UiscsCrh6ISDsB4WKad+7YfbQ9Te/7pnVo7QmnZkNokYBKCDyQLlgjiqW19wGfvEIyS9X/non1XKGaQrpIqJDMTID/fb24/Lx5jqesC4v6M+ERSrWsK7eErcz SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2018 10:02:54.2403 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3722c8de-99ef-4881-7910-08d62f60b6d6 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4676 Subject: Re: [dpdk-dev] [PATCH 3/5] common/dpaax: add library for PA VA translation table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Oct 2018 10:02:56 -0000 On Thursday 11 October 2018 02:33 PM, Burakov, Anatoly wrote: > On 09-Oct-18 11:45 AM, Shreyansh Jain wrote: >> On Tuesday 25 September 2018 07:09 PM, Shreyansh Jain wrote: >>> Hello Anatoly, >>> >>> On Tuesday 25 September 2018 06:58 PM, Burakov, Anatoly wrote: >>>> On 25-Sep-18 1:54 PM, Shreyansh Jain wrote: >>>>> A common library, valid for dpaaX drivers, which is used to maintain >>>>> a local copy of PA->VA translations. >>>>> >>>>> In case of physical addressing mode (one of the option for FSLMC, and >>>>> only option for DPAA bus), the addresses of descriptors Rx'd are >>>>> physical. These need to be converted into equivalent VA for rte_mbuf >>>>> and other similar calls. >>>>> >>>>> Using the rte_mem_virt2iova or rte_mem_virt2phy is expensive. This >>>>> library is an attempt to reduce the overall cost associated with >>>>> this translation. >>>>> >>>>> A small table is maintained, containing continuous entries >>>>> representing a continguous physical range. Each of these entries >>>>> stores the equivalent VA, which is fed during mempool creation, or >>>>> memory allocation/deallocation callbacks. >>>>> >> >> [...] >> >>> >>>> >>>> Also, a couple of nitpicks below. >>>> >>>>>   cosnfig/common_base                            |   5 + >>>>>   config/common_linuxapp                        |   5 + >>>>>   drivers/common/Makefile                       |   4 + >>>>>   drivers/common/dpaax/Makefile                 |  31 ++ >>>>>   drivers/common/dpaax/dpaax_iova_table.c       | 509 >>>>> ++++++++++++++++++ >>>>>   drivers/common/dpaax/dpaax_iova_table.h       | 104 ++++ >>>>>   drivers/common/dpaax/dpaax_logs.h             |  39 ++ >>>>>   drivers/common/dpaax/meson.build              |  12 + >>>> >>>> >>>> >>>>> +    DPAAX_DEBUG("Add: Found slot at (%"PRIu64")[(%zu)] for >>>>> vaddr:(%p)," >>>>> +            " phy(%"PRIu64"), len(%zu)", entry[i].start, e_offset, >>>>> +            vaddr, paddr, length); >>>>> +    return 0; >>>>> +} >>>>> + >>>>> +int >>>>> +dpaax_iova_table_del(phys_addr_t paddr, size_t len __rte_unused) >>>> >>>> len is not unused. >>> >>> I will fix this. >>> Actually, this function itself is useless - more for symmetry reason. >>> Callers would be either simply updating the table, or ignoring it >>> completely. But, yes, this is indeed wrong that I set that unused. >>> >> >> Actually, I was wrong in my first reply. In case of >> dpaax_iova_table_del(), len is indeed redundant. This is because the >> mapping is for a complete page (min of 2MB size), even if the request >> is for lesser length. So, removal of a single entry (of fixed size) >> would be done. >> >> In fact, while on this, I think deleting a PA->VA entry itself is >> incorrect (not just useless). A single entry (~2MB equivalent) can >> represent multiple users (working on a rte_malloc'd area, for >> example). So, effectively, its always an update - not an add or del. > > I'm not sure what you mean here. If you got a mem event about memory > area being freed, it's guaranteed to *not* have any users - neither > malloc, nor any other memory. And len is always page-aligned. ok. Maybe I am getting this wrong, but consider this: 1) hugepage size=2MB 2) a = malloc(1M) this will pin an entry in table for a block starting at VA=(a) and PA=(a'). Each entry is of 2MB length - that means, even if someone were to access a+1048577 for an equivalent PA, they would get it (though, that is a incorrect access). 3) b = malloc(1M) this *might* lead to a case where same 2MB page is used and VA=(b==(a+1MB)). Being hugepage backed, PA=(b=PA(a)+1M). = After b, the PA-VA table has a single entry of 2MB, representing two mallocs. It can be used for translation for any thread requesting PAs of a or b. 4) Free(a) - this would attempt to remove one 2MB entry from PA-VA table. But, 'b' is already valid. Access to get_pa(VA(b)) should return me the PA(b). - 'len' is not even used as the entry in PA-VA table is of a fixed size. In the above, (3) is an assumption I am making based on my understanding how mem allocator is working. Is that wrong? Basically, this is a restriction of this table - it has a min chunk of 2MB - even for 1G hugepages - and hence, it is not possible to honor deletes. I know this is convoluted logic - but, this keeps it simple and use-able without much performance impact. [...]