From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1EF55A0032 for ; Fri, 18 Feb 2022 13:41:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16EAC40150; Fri, 18 Feb 2022 13:41:47 +0100 (CET) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mails.dpdk.org (Postfix) with ESMTP id 97E7640141 for ; Fri, 18 Feb 2022 13:41:45 +0100 (CET) Received: by mail-wm1-f53.google.com with SMTP id az26-20020a05600c601a00b0037c078db59cso6314196wmb.4 for ; Fri, 18 Feb 2022 04:41:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UiaQmogWgDXVyNoD9MjJStSurbfnOzC/S/mZ/Ddx15E=; b=Wstduln2HtakSX+eDqqGIZB+Jr+CIWKZjdjpDtkq6+vKfAUIO8EMvQsMHiN8pCF1Vi mQ7WacyOEy4Pz5mqKohhbJQ+YpP5CTB/JSlC2mgNUW9D5bfeFjDpWFD4kibl2LpL+q1C OEMqsmIIYzmUYK9NP6bg5V+3AZqQpMURq6hrvlxX93OCjG4ORKl9TMNHZ9eLyHhNbhJv VBX690GQ5ip2Nnc376fCHEWl7cmr/kRn3T7bZkpW7biEFspTn9sTMLZerj99fT2VsjB4 xUtpH7XQSHioHMIPEBkmMiu0O0Ax8YONos9eLnx0eeQ4lgPbvgWoJKHQ250rcAyjcrMP tz1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UiaQmogWgDXVyNoD9MjJStSurbfnOzC/S/mZ/Ddx15E=; b=WfEPJmBhyTWvpZZgNO1aNn20P7pjzrJRolgg1WZ2VQmbgQN30/mD2gWs3JiC59+NdB xpLmfgZ8nPqEtbK09viEFUtJrqU0k0aJT4uFM+ipm1czD2uDa8iioQpqWoixwDd0xrMo hurvM1xSff4OCVBuUGzUUdxRozeCgI4S6SvoW7/i3sn3niP7MGgwm7Q6B8eowstWNe5Q t9HlPJ19E+kTQCZMIHWmbWl46PreGoQ+0IHWNAluYSV4e8G3Zj0amdvpgJvcA05KIkR5 Jh1me19r86U+muH0uOJqeMYkVDm5044QQ4J3TIoiSpB/IckySmr4rtgmJDuGFG/cyqaQ tvUA== X-Gm-Message-State: AOAM531Lz75wD/Ga8iPAKRCVpSJ8q590bha3zkt/wJiB082yLZxBSfuh SaI/j2ad0Dd1+VXp+bnon+o= X-Google-Smtp-Source: ABdhPJxlBMvQSLmBUnQ6Y+QdiE9NJaooQTOgvlJ9SL+Io3IGS4m7LiPn8WmV7fOdm/CcXy83OSImJg== X-Received: by 2002:a1c:7704:0:b0:354:4d2b:9d5 with SMTP id t4-20020a1c7704000000b003544d2b09d5mr10436921wmi.155.1645188105330; Fri, 18 Feb 2022 04:41:45 -0800 (PST) Received: from localhost ([2a01:4b00:f41a:3600:360b:9754:2e3a:c344]) by smtp.gmail.com with ESMTPSA id b18sm33810273wrx.92.2022.02.18.04.41.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Feb 2022 04:41:44 -0800 (PST) From: luca.boccassi@gmail.com To: Dmitry Kozlyuk Cc: Viacheslav Ovsiienko , dpdk stable Subject: patch 'app/testpmd: fix external buffer allocation' has been queued to stable release 20.11.5 Date: Fri, 18 Feb 2022 12:38:10 +0000 Message-Id: <20220218123931.1749595-41-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220218123931.1749595-1-luca.boccassi@gmail.com> References: <20220218123931.1749595-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 20.11.5 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/20/22. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/a5f8244a5f867f9dc4b816d978af7e9babdf597f Thanks. Luca Boccassi --- >From a5f8244a5f867f9dc4b816d978af7e9babdf597f Mon Sep 17 00:00:00 2001 From: Dmitry Kozlyuk Date: Fri, 17 Dec 2021 11:58:16 +0200 Subject: [PATCH] app/testpmd: fix external buffer allocation [ upstream commit 13b196425ce371a82fa8008fa46ce66861d4202b ] External pinned buffer memory (--mp-alloc=xbuf) was allocated as multiple IOVA-contiguous memzones of 2M size and 2M alignment. Due to the malloc overhead and the alignment requirement, each 2M memzone consumed 4M of hugepage memory: 2M of usable memory + X of malloc overhead + (2M-X) padding. The allocation often failed with 2M hugepages and IOVA-as-PA if a PA-contiguous span of 2 hugepages could not be found. Also, with any hugepage size and IOVA mode memory consumption was almost 2x of the usable amount. Alignment requirement of 2M for external buffers is redundant. It was an attempt to ensure IOVA-contiguity by forcing memzones to start at hugepage boundaries, while 2M size intended to leave no unused space on the page. As shown above, this in fact caused excessive memory consumption and decreased the chance of a successful allocation. RTE_MEMZONE_F_IOVA_CONTIG already ensures IOVA-contiguity. Remove the alignment requirement. Reduce the memzone size by the malloc overhead size (4 cache lines), so that memory consumption for each memzone is (2M-X) of usable memory + X of malloc overhead = 2M. This also means that whenever there are free 2M hugepages, an IOVA-contiguous memzone can always be allocated. Fixes: 72512e1897b2 ("app/testpmd: add mempool with external data buffers") Signed-off-by: Dmitry Kozlyuk Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/testpmd.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 697e79b2fc..7e4b96d442 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -78,7 +78,13 @@ #endif #define EXTMEM_HEAP_NAME "extmem" -#define EXTBUF_ZONE_SIZE RTE_PGSIZE_2M +/* + * Zone size with the malloc overhead (max of debug and release variants) + * must fit into the smallest supported hugepage size (2M), + * so that an IOVA-contiguous zone of this size can always be allocated + * if there are free 2M hugepages. + */ +#define EXTBUF_ZONE_SIZE (RTE_PGSIZE_2M - 4 * RTE_CACHE_LINE_SIZE) uint16_t verbose_level = 0; /**< Silent by default. */ int testpmd_logtype; /**< Log type for testpmd logs */ @@ -926,12 +932,11 @@ setup_extbuf(uint32_t nb_mbufs, uint16_t mbuf_sz, unsigned int socket_id, ext_num = 0; break; } - mz = rte_memzone_reserve_aligned(mz_name, EXTBUF_ZONE_SIZE, - socket_id, - RTE_MEMZONE_IOVA_CONTIG | - RTE_MEMZONE_1GB | - RTE_MEMZONE_SIZE_HINT_ONLY, - EXTBUF_ZONE_SIZE); + mz = rte_memzone_reserve(mz_name, EXTBUF_ZONE_SIZE, + socket_id, + RTE_MEMZONE_IOVA_CONTIG | + RTE_MEMZONE_1GB | + RTE_MEMZONE_SIZE_HINT_ONLY); if (mz == NULL) { /* * The caller exits on external buffer creation -- 2.30.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2022-02-18 12:37:39.675115810 +0000 +++ 0041-app-testpmd-fix-external-buffer-allocation.patch 2022-02-18 12:37:37.642790848 +0000 @@ -1 +1 @@ -From 13b196425ce371a82fa8008fa46ce66861d4202b Mon Sep 17 00:00:00 2001 +From a5f8244a5f867f9dc4b816d978af7e9babdf597f Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 13b196425ce371a82fa8008fa46ce66861d4202b ] + @@ -33 +34,0 @@ -Cc: stable@dpdk.org @@ -42 +43 @@ -index 6c387bde84..e1da961311 100644 +index 697e79b2fc..7e4b96d442 100644 @@ -45 +46 @@ -@@ -84,7 +84,13 @@ +@@ -78,7 +78,13 @@ @@ -60 +61 @@ -@@ -1061,12 +1067,11 @@ setup_extbuf(uint32_t nb_mbufs, uint16_t mbuf_sz, unsigned int socket_id, +@@ -926,12 +932,11 @@ setup_extbuf(uint32_t nb_mbufs, uint16_t mbuf_sz, unsigned int socket_id,