From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2F40141CE5; Mon, 20 Feb 2023 11:59:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E024743017; Mon, 20 Feb 2023 11:59:26 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 040AC40395 for ; Mon, 20 Feb 2023 11:59:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676890765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rgcdGhvpyhG+QkJ2x3fEDsQN+OcytVya4VJ8yCG319g=; b=CCbcRxmQZi71Qj8S/1wylaSl3NhLML54h677LDhoOLCWJ5/DC5DmDT6OKNoHAqwyms2Ks9 FOwRAhZabBM2LHHUOkKV9O3VaoeTcIVHRSxsxf1E4/SaTfEv8AbOeHCJs5+V0AAn75f9kR Y5d/n6RA0Z3PiugPSxwphwg9CCipnGA= Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-280-3W_R80snM3-qDAafeTbRSQ-1; Mon, 20 Feb 2023 05:59:24 -0500 X-MC-Unique: 3W_R80snM3-qDAafeTbRSQ-1 Received: by mail-pl1-f197.google.com with SMTP id k17-20020a170902d59100b0019abcf45d75so202961plh.8 for ; Mon, 20 Feb 2023 02:59:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rgcdGhvpyhG+QkJ2x3fEDsQN+OcytVya4VJ8yCG319g=; b=gB28AiJbLdgRZKBJDpPCgTZ3vzkDNZHqDPdBczyVjy7B/ziATRCQ6nBHxSJIKG+A72 ncTRn6VW2gfaQ4f20ji1yCZRsksmvkg8mQuE+Dyrvm62Tat+65J/HL2L9GY4WCfb6mKU kdOPNQoMeQKjdrG0whKnIY4QBm0kEYI5dCTFVWcdiOE4iVR1OXmyNlc2fRyKfF+1b1XR zLnFDX6SOGFBKrIvDRg/KxE1C85BieK1RMkpActE5Fd9fs+NHv2MqVo0QVjKKosOSRFe IRG01AnKwnyh+dFTuJC3Q9bybArU4/8j6dn25leFSFfRkfeUPzn1Fa62CBxfDFJMtK3l 0wjg== X-Gm-Message-State: AO0yUKXBuAOZdiBNLfuhc5yOKGc4NDW0lyTx36hDslt6htcw4oYoAk1q fGqU434PIOYhzay7jEvtSLSGZFE5WUqomQVroqNUBa7NxcA9UqlD0kXcgTBcX3h0R7QvdCHSiXo zclFseJq1stkJ0qVGmyY= X-Received: by 2002:a17:90b:278d:b0:236:70c2:9819 with SMTP id pw13-20020a17090b278d00b0023670c29819mr480356pjb.116.1676890762952; Mon, 20 Feb 2023 02:59:22 -0800 (PST) X-Google-Smtp-Source: AK7set9sHhiji61IIOET/Q7elhG+KGTvwWTBGjvsq9j3Y5MHEksrue053VfXcL2u1tGNkuTzKxMbOkpi+u/oOo9Xsuc= X-Received: by 2002:a17:90b:278d:b0:236:70c2:9819 with SMTP id pw13-20020a17090b278d00b0023670c29819mr480349pjb.116.1676890762592; Mon, 20 Feb 2023 02:59:22 -0800 (PST) MIME-Version: 1.0 References: <20230210063022.52171-1-changfengnan@bytedance.com> In-Reply-To: <20230210063022.52171-1-changfengnan@bytedance.com> From: David Marchand Date: Mon, 20 Feb 2023 11:59:11 +0100 Message-ID: Subject: Re: [PATCH] malloc: fix malloc performance may becomes worse as the number of malloc increases To: Fengnan Chang Cc: anatoly.burakov@intel.com, rsanford@akamai.com, dev@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= , Liang Ma , Stephen Hemminger X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Feb 10, 2023 at 7:30 AM Fengnan Chang wrote: > > Here is a simple test case: > " > uint64_t entry_time, time; > size_t size =3D 4096; > unsigned align =3D 4096; > for (int j =3D 0; j < 10; j++) { > entry_time =3D rte_get_timer_cycles(); > for (int i =3D 0; i < 2000; i++) { > rte_malloc(NULL, size, align); > } > time =3D (rte_get_timer_cycles()-entry_time) * 1000000 / > rte_get_timer_hz(); > printf("total open time %lu avg time %lu\n", time, time/2000); > } > " > > Single rte_malloc cost time may becomes wrose as the number of malloc > increases, In my env, first round avg time is 15us, second is 44us, > third is 77us, fourth is 168us... > > The reason is,in the malloc process, malloc_elem_alloc may split new_elem > if there have too much free space after new_elem, and insert the trailer > into freelist. When alloc 4k with align 4k, the trailer very likely inser= t > to free_head[2] again, it makes free_head[2] longer. when alloc 4k again, > it will search free_head[2] from begin, with the number of malloc increas= es, > search free_head[2] need more time, so the performance will become worse. > Same problem will also occurs in alloc 64k with align 64k, but if alloc > 4k with align 64, doesn't have this problem. > > Fix this by adjust free_head list size range, make free_head[3] hold > elements which bigger or equal 4k, free_head[4] hold elements which bigge= r > or equal 16k. > In terms of probabilities, when alloc 4k or 16k, the probability of findi= ng > a suitable elem from a larger size list is greater than from a smaller > size list. > > Signed-off-by: Fengnan Chang Acked-by: Morten Br=C3=B8rup The change looks simple enough. I see an improvement with the (verbatim) malloc_perf_autotest unit tests for 1M allocations too. Let's take this change now and see how it goes with -rc1 testing. Applied, thanks. --=20 David Marchand