From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DD14A0544; Mon, 10 Oct 2022 17:15:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4EBEE427F6; Mon, 10 Oct 2022 17:15:46 +0200 (CEST) Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by mails.dpdk.org (Postfix) with ESMTP id 933F34021E for ; Mon, 10 Oct 2022 17:15:45 +0200 (CEST) Received: by mail-lf1-f48.google.com with SMTP id j4so17174538lfk.0 for ; Mon, 10 Oct 2022 08:15:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=p221XBH0MF3DswPxJrnK12b0F3w6lTesX6x788UaxAE=; b=GsqJguM4mD8fFlOCEGTo7jneWf1m23jqqpIcLekzn1owqT4Qi/YH0KikjB1KyqgS3u Wd/GNzWIJp5UYq9ZXnKIb/AJQrKDyTIfRZ6pNbmKRvLVST/2R4YJMOuE21VKZukIc4W+ nTNajp7B/1oRK4S/JQ8vYJj0mUjBkl0w7hoGQQK8RFwj3KdqlpKCtlnFp3ogh0mSYwWf AuAp0mk2iPZ7VOZRSTJxBGLK6SwM1782OzlYnkKy/qGxAHifbugPdFzzbchXnyI+y6gB +hQxOJFZLdCaHKV9N3/miy3yXvqoMgdX55QBrnQ71H94yyGk0CVcp5TwwY6oXRGIAlkB xgjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p221XBH0MF3DswPxJrnK12b0F3w6lTesX6x788UaxAE=; b=pCM/O8mX/VPAXiIWK9g/KXYWr8hO23vo2cj1fd0q9HoIDU2GUL8KF0DzeA4EgXczNx zeuGiEiGK+RHbYNpL4rG7E77n+Ry7PhSlqrkRZkWu8Go7uw9fEV2DUOeb23sQPSLqS2+ 4d3lp4Za1CQXMO+6BQh8Z30xFZEIXYxrsBMW+Mpz045ayCfu3HiLGN0b9M109zp+BZLC KFigUzWJA1+QvlgQOYUcGTKqRCbtFQIOgTElfyNZcablKa1und3WMuVEILDhc7Pzrnx+ S9GV+BomF/hHDt1LW/GENbZ8BNfuj3x6pyxRCNp9UtupjKUYS9ukrg/jlXGtJDm9wgOI fR/g== X-Gm-Message-State: ACrzQf3Y5WEHVpy7ExYtx19P8vOdyW8RF1zvC0OSa3IOMiyD7Zp+Xpvi evxHOn6aMtcmYRe4AvJU9cE= X-Google-Smtp-Source: AMsMyM77VSViDFZ1vOMEhXWzs59EMRnXExQlXcPr3EX8Kv3onQgz8WVdVNEJ7f6GuRwkyki4p9NFDw== X-Received: by 2002:ac2:5f84:0:b0:4a2:38e2:9682 with SMTP id r4-20020ac25f84000000b004a238e29682mr6547942lfe.142.1665414943365; Mon, 10 Oct 2022 08:15:43 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id h4-20020a056512220400b004994117b0fdsm1448515lfu.281.2022.10.10.08.15.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Oct 2022 08:15:42 -0700 (PDT) Date: Mon, 10 Oct 2022 18:15:41 +0300 From: Dmitry Kozlyuk To: "Umakiran Godavarthi (ugodavar)" Cc: "anatoly.burakov@intel.com" , "dev@dpdk.org" , "stephen@networkplumber.org" Subject: Re: DPDK 19.11.5 Legacy Memory Design Query Message-ID: <20221010181541.2fb4923e@sovereign> In-Reply-To: References: <20220922120052.710c2cd1@sovereign> <20220923144727.155be566@sovereign> <20220923161057.0205e703@sovereign> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Umakiran, Please quote what is needed and reply below the quotes. 2022-09-26 13:06 (UTC+0000), Umakiran Godavarthi (ugodavar): > Hi Dimitry >=20 > We know If the application does unmap, DPDK native heaps are not getting = cleaned up in =E2=80=9Cnative: use regular DPDK memory=E2=80=9D memory ty= pe. >=20 > Can we get an API from DPDK community please to clean up the heaps given = the VA and length to be freed ? Like below >=20 > https://doc.dpdk.org/api/rte__memory_8h.html#afb3c1f8be29fa15953cebdad3a9= cd8eb >=20 > int rte_extmem_unregister > ( > void * > va_addr, > size_t > len > ) >=20 > Unregister external memory chunk with DPDK. > Note > Using this API is mutually exclusive with rte_malloc family of API's. > This API will not perform any DMA unmapping. It is expected that user wil= l do that themselves. > Before calling this function, all other processes must call rte_extmem_de= tach to detach from the memory area. > Parameters > va_addr > Start of virtual area to unregister > len > Length of virtual area to unregister > Returns > =C2=B7 0 on success > =C2=B7 -1 in case of error, with rte_errno set to one of the foll= owing: EINVAL - one of the parameters was invalid ENOENT - memory chunk was= not > If we get such API in native way , it will be good for us instead of chan= ging the design and going all the way writing code for new memory type >=20 > Please suggest us can we get an API to clean up DPDK NATIVE POOLS if VA_A= DDR and LEN is given ? 1. Let's clarify the terms first. Malloc heaps which store hugepages that an application can allocate. These are internal structures of the DPDK memory manager. Correct, unmapping memory without maintaining these structures leads to the issue you're trying to solve. Pools are allocated on top of some kind of memory. Note that "native", "xmem", etc. are specific to TestPMD app. To DPDK, "native" means memory from the DPDK allocator (pool memory can also come from outside of DPDK for example). 2. DPDK 19.11 is LTS and will be EOL soon [1]. New APIs are only added to the upstream. 3. A new API needs rationale. I still don't see why you need legacy mode in your case. New API also would not fit well into the legacy mode idea: static memory layout. In dynamic memory mode, it would be useless, because unneeded pages are not allocated in the first place and freed once not used anymore. [1]: https://core.dpdk.org/roadmap/#stable > From: Umakiran Godavarthi (ugodavar) > Date: Monday, 26 September 2022 at 6:25 PM > To: Dmitry Kozlyuk > Cc: anatoly.burakov@intel.com , dev@dpdk.org <= dev@dpdk.org>, stephen@networkplumber.org > Subject: Re: DPDK 19.11.5 Legacy Memory Design Query > Thanks @Dmitry Kozlyuk for your suggesti= ons >=20 > I will try the following for DPDK pool creation >=20 > My logic of calculating MBUF=E2=80=99s remains same >=20 > Saw this code in DPDK testpmd where External heap memory is used >=20 > case MP_ALLOC_XMEM_HUGE: > { > int heap_socket; > bool huge =3D mp_alloc_type =3D=3D MP_ALLOC_XMEM_= HUGE; >=20 > if (setup_extmem(nb_mbuf, mbuf_seg_size, huge) < = 0) > rte_exit(EXIT_FAILURE, "Could not create = external memory\n"); >=20 > heap_socket =3D > rte_malloc_heap_get_socket(EXTMEM_HEAP_NA= ME); > if (heap_socket < 0) > rte_exit(EXIT_FAILURE, "Could not get ext= ernal memory socket ID\n"); >=20 > TESTPMD_LOG(INFO, "preferred mempool ops selected= : %s\n", > rte_mbuf_best_mempool_ops()); > rte_mp =3D rte_pktmbuf_pool_create(pool_name, nb_= mbuf, > mb_mempool_cache, 0, mbuf_seg_siz= e, > heap_socket); > break; > } >=20 > So I will do the same >=20 >=20 > 1. EAL Init > 2. Calculate MBUF we need for our application > 3. Then create pool using MP_ALLOC_XEM_HUGE >=20 > 1,2,3 should work right , that should avoid heap corruption issues right ? Yes, this snippet (and relevant functions there) is for your case. >=20 > We have total 3 types pool creation >=20 > /* > * Select mempool allocation type: > * - native: use regular DPDK memory > * - anon: use regular DPDK memory to create mempool, but populate using > * anonymous memory (may not be IOVA-contiguous) > * - xmem: use externally allocated hugepage memory > */ >=20 > Instead of Freeing unused virtual memory in DPDK native way, we just crea= te a pool with type XMEM_HUGE and add page by page to it like testpmd code >=20 > Please let me know 1,2,3 steps are good right, so we need to boot DPDK wi= th socket mem >=20 > --socket-mem 2048 so that DPDK takes only 1 page natively and boots up ri= ght ? --socket-mem is in megabytes, so it's --socket-mem 2 and 2MB hugepages must be available. Maybe even --no-huge will suit your case since you allocate all hugepages yourself effectively.