From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE3EB42BD2; Mon, 29 May 2023 13:21:06 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 68E12410DD; Mon, 29 May 2023 13:21:06 +0200 (CEST) Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by mails.dpdk.org (Postfix) with ESMTP id 3A280410D7 for ; Mon, 29 May 2023 13:21:04 +0200 (CEST) Received: by mail-qt1-f170.google.com with SMTP id d75a77b69052e-3f6b2bb393cso3772151cf.0 for ; Mon, 29 May 2023 04:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1685359264; x=1687951264; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=PuwsS6AdsuIC/0o5GF/YiGoIxYw4tWLV/g34CyT5Fas=; b=g6GV117jvPXYmMn7PO4fcdXWtEv4soZf1E8Yf160eU0cyMt3HjLv7YXD/laiUQ576I 3aEQN2ecHY5JM30+wD7hu4h5Zj13txKNcVTatNoL06TRoQWpvTjzIjBdkgOBsQcVMVQ6 uYrvbnA2R73HymbCK76WGuosUdLMS7r64WX4rWGG+3gqhLnLBc2p0DbeFgmv8kQ0nktk jOWxzgTLtAjLFqasJEEnV45wR/jqy2rZjIVLcZ2iGoleKIFZ8dRiVzeogaAsPSYkdgcR dMhv9CJYKP1fZgeU9OX9iLQjKzjn8Z1lRbAU4sP5RFtrVGI6nvlTPBSBO0ibxZT9Iqof aPhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685359264; x=1687951264; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PuwsS6AdsuIC/0o5GF/YiGoIxYw4tWLV/g34CyT5Fas=; b=fRa3ry1lCuReLaH2qHiVonTzU6rNCjDuvsbqJv/xpqXYptYzoKjHPM+wplc/dSnige z5BYm9URtfSkkutdkqtZVzUZI/167qAAy6+bqKrAiv5/GIrvONBaZrvLIjICV8Et3lpL wvPwbMzdfpghEK2/+LND9sl0AahIl7DvCLVhUNp1qdJDXIa9hr+0t08eQ45wdZ75P/vB mkvrhNPXQY73PTQYgxNkr+YKT53k0Gxgej0WyRNhTlMCcjaTxaSIdoQCVmZN5Jn2qVXF A6JhiYrHiqnbAGv98BiEawae568IWybVEkZCDphSkLVOIYmx64RIxQIgGP/1fTrh1eLb AeGg== X-Gm-Message-State: AC+VfDyMtD2fwP7aXMi1KG6df+e+9aRb2lhCvsJ3K1zwlmFYC/OtP3TM b5lL943MP8uBDHQ6Oyna79dPyIvb63UpuCmQxXjrQA== X-Google-Smtp-Source: ACHHUZ5Pxkpc3VLZ2VhFoA4YwYjWT1xVpxRUOmnsuTtgLliq3mF978w3HjvMPYxLUDQPD/8+jb3MxdWApfoIkrhrPOg= X-Received: by 2002:a05:622a:1a22:b0:3f6:a9a2:3b66 with SMTP id f34-20020a05622a1a2200b003f6a9a23b66mr11328868qtb.5.1685359264285; Mon, 29 May 2023 04:21:04 -0700 (PDT) MIME-Version: 1.0 References: <20230526034127.18231-1-changfengnan@bytedance.com> <4099387b-9f8a-ea0c-8ab9-4c9de15117c4@intel.com> <77e7da3f-6a4c-abf9-bffc-6af2f59e1830@intel.com> In-Reply-To: <77e7da3f-6a4c-abf9-bffc-6af2f59e1830@intel.com> From: Fengnan Chang Date: Mon, 29 May 2023 19:20:53 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v3] eal: fix eal init may failed when too much continuous memsegs under legacy mode To: "Burakov, Anatoly" Cc: dev@dpdk.org, Lin Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Burakov, Anatoly =E4=BA=8E2023=E5=B9=B45=E6=9C= =8826=E6=97=A5=E5=91=A8=E4=BA=94 23:33=E5=86=99=E9=81=93=EF=BC=9A > > On 5/26/2023 3:41 PM, Burakov, Anatoly wrote: > > On 5/26/2023 4:41 AM, Fengnan Chang wrote: > >> Under legacy mode, if the number of continuous memsegs greater > >> than RTE_MAX_MEMSEG_PER_LIST, eal init will failed even though > >> another memseg list is empty, because only one memseg list used > >> to check in remap_needed_hugepages. > >> Fix this by make remap_segment return how many segments mapped, > >> remap_segment try to map most contiguous segments it can, if > >> exceed it's capbility, remap_needed_hugepages will continue to > >> map other left pages. > >> > >> For example: > >> hugepage configure: > >> cat > >> /sys/devices/system/node/node*/hugepages/hugepages-2048kB/nr_hugepages > >> 10241 > >> 10239 > >> > >> startup log: > >> EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 > >> EAL: Detected memory type: socket_id:1 hugepage_sz:2097152 > >> EAL: Creating 4 segment lists: n_segs:8192 socket_id:0 > >> hugepage_sz:2097152 > >> EAL: Creating 4 segment lists: n_segs:8192 socket_id:1 > >> hugepage_sz:2097152 > >> EAL: Requesting 13370 pages of size 2MB from socket 0 > >> EAL: Requesting 7110 pages of size 2MB from socket 1 > >> EAL: Attempting to map 14220M on socket 1 > >> EAL: Allocated 14220M on socket 1 > >> EAL: Attempting to map 26740M on socket 0 > >> EAL: Could not find space for memseg. Please increase 32768 and/or > >> 65536 in > >> configuration. > >> EAL: Couldn't remap hugepage files into memseg lists > >> EAL: FATAL: Cannot init memory > >> EAL: Cannot init memory > >> > >> Signed-off-by: Fengnan Chang > >> Signed-off-by: Lin Li > >> Signed-off-by: Burakov Anatoly > > > > Hi, > > > > Thanks for taking my suggested implementation on board! > > > >> --- > >> lib/eal/linux/eal_memory.c | 55 +++++++++++++++++++++++++-----------= -- > >> 1 file changed, 36 insertions(+), 19 deletions(-) > >> > >> diff --git a/lib/eal/linux/eal_memory.c b/lib/eal/linux/eal_memory.c > >> index 60fc8cc6ca..085defdee5 100644 > >> --- a/lib/eal/linux/eal_memory.c > >> +++ b/lib/eal/linux/eal_memory.c > >> @@ -681,6 +681,7 @@ remap_segment(struct hugepage_file *hugepages, int > >> seg_start, int seg_end) > >> /* find free space in memseg lists */ > >> for (msl_idx =3D 0; msl_idx < RTE_MAX_MEMSEG_LISTS; msl_idx++) { > >> + int free_len; > >> bool empty; > >> msl =3D &mcfg->memsegs[msl_idx]; > >> arr =3D &msl->memseg_arr; > >> @@ -692,24 +693,28 @@ remap_segment(struct hugepage_file *hugepages, > >> int seg_start, int seg_end) > >> /* leave space for a hole if array is not empty */ > >> empty =3D arr->count =3D=3D 0; > >> - ms_idx =3D rte_fbarray_find_next_n_free(arr, 0, > >> - seg_len + (empty ? 0 : 1)); > >> - > >> - /* memseg list is full? */ > >> - if (ms_idx < 0) > >> + /* find start of the biggest contiguous block and its size */ > >> + ms_idx =3D rte_fbarray_find_biggest_free(arr, 0); > >> + free_len =3D rte_fbarray_find_contig_free(arr, ms_idx); > >> + if (free_len < 0) > >> continue; > > Missed this. > > When we're splitting up segments, we're looking for contiguous free > areas that are at least two memsegs long, so if this memseg is full but > contains segments that were split up, there will be a bunch of holes in > it, and free_len will be 1 (because every hole will be 1 segment long by > definition). So, we should not accept anything that is at least two > segments long. Got it, thanks for your reminder. > > -- > Thanks, > Anatoly >