From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0BB3A00C2; Sun, 30 Oct 2022 17:13:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CEF0142B83; Sun, 30 Oct 2022 17:13:02 +0100 (CET) Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by mails.dpdk.org (Postfix) with ESMTP id C6C3542B80 for ; Sun, 30 Oct 2022 17:13:01 +0100 (CET) Received: by mail-pj1-f44.google.com with SMTP id b11so8534529pjp.2 for ; Sun, 30 Oct 2022 09:13:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=Njd1KblC5JH5wd5Z/+ProlOawWdZsnxEHm1dvkeP6YM=; b=FaeYvgcNxv0JnOTYpUlciQin95Yn7ehjbTbV0FW7h/JUUQUVAZU1MwTMCrZpFCfPiM Cq9yum8e3YYJTq65P0eGh+8J/DdaWiwk02gVlIm9P/lzMbHfiZoc58UXRRetAqO+TIbI zX8azrRs2TIrUlbhhrWihFDzPzFuGfY54G6JrJJ9pb4zattFeONOgRkrIDZajagmqVmk 9gaw73Kvako3uRB1XmMtAZYY5+qTI3CwilUwUGx5Z/UTzNZdpCg+R/9gftNVLK7/RqU7 4vtki+vDFbdJq+VCvkYoVpgF2F4KtaL2eqj/CruQwD4jSAcsqTOO44pf4YgNztFMXWsz iyfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Njd1KblC5JH5wd5Z/+ProlOawWdZsnxEHm1dvkeP6YM=; b=eo6k7Yzko7YSeBEIEsieFFEjl0xQQoOmqoC7if+epCvIS8DyDA2CykbwC6vsewQRUv n3wtBvKr2IqA0WHaQHPBtzb0KG2G1FNT+RtjaFpVdA1ZPKdEGycNdIZ5Pq9MVBKSrgsA TF9+8cInuag+iO0gVeR0phwntyF0Rxl94Gft2/tSrJKaLJR4TRARwpflVG714k95G6Hh oG4tdOfc2rEApt4nhYqt/JI5zoZUyL4lIGcoYpluOPMlsBUNJtPQnHtUmya6OUyhp/PM AdcbN8eVha9RoW3ZNkCflxUSn0SMOjVBVzs1UjPbAj+lWnliUS9fGCNgZd65bP/z9Pi+ VKog== X-Gm-Message-State: ACrzQf3rxRFCukldMQF6UnnUbl5EpEtGT9fJ3cFHZ/WoaMz8a8IS2hMa Ba44l8oCYrA+0zB6b9Cn2EFN5A== X-Google-Smtp-Source: AMsMyM5YL7gB1s1jb7lCxD+u7uX1fepKoP2vv7jeF1rJcWWIKR7X0amUOwCWowfmVHKxgaimeYMnAA== X-Received: by 2002:a17:90a:2847:b0:213:9ae5:b9b4 with SMTP id p7-20020a17090a284700b002139ae5b9b4mr13201963pjf.136.1667146381016; Sun, 30 Oct 2022 09:13:01 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id o6-20020a170902bcc600b0018703bf42desm2859433pls.159.2022.10.30.09.13.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 09:13:00 -0700 (PDT) Date: Sun, 30 Oct 2022 09:12:59 -0700 From: Stephen Hemminger To: Morten =?UTF-8?B?QnLDuHJ1cA==?= Cc: , , Subject: Re: [PATCH] mempool: split statistics from debug Message-ID: <20221030091259.21b7148d@hermes.local> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87463@smartserver.smartshare.dk> References: <20221030115445.2115-1-mb@smartsharesystems.com> <98CBD80474FA8B44BF855DF32C47DC35D87463@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, 30 Oct 2022 15:04:18 +0100 Morten Br=C3=B8rup wrote: > > From: Morten Br=C3=B8rup [mailto:mb@smartsharesystems.com] > > Sent: Sunday, 30 October 2022 12.55 > >=20 > > Split statistics from debug, to make mempool statistics available > > without > > the performance cost of continuously validating the cookies in the > > mempool > > elements. =20 >=20 > mempool_perf_autotest shows that the rate_persec drops to a third (-66 %)= when enabling full mempool debug - quite prohibitive! >=20 > With this patch, the performance cost is much lower. I don't have detaile= d test results, but the initial tests without mempool cache show a performa= nce drop of -11 %. Significant, but not prohibitive. One trick to avoid conditional in fast path would be to add a dummy stats[]= per core. Another would be to move the fast path get/put stats into the mempool_cache. The current model has stats[] per cpu always on another cache line. diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h index 1f5707f46a21..87905b7286a6 100644 --- a/lib/mempool/rte_mempool.h +++ b/lib/mempool/rte_mempool.h @@ -236,8 +236,10 @@ struct rte_mempool { struct rte_mempool_memhdr_list mem_list; /**< List of memory chunks= */ =20 #ifdef RTE_LIBRTE_MEMPOOL_DEBUG - /** Per-lcore statistics. */ - struct rte_mempool_debug_stats stats[RTE_MAX_LCORE]; + /** Per-lcore statistics. + * Allocate one additional per-cpu slot for non-DPDK threads + */ + struct rte_mempool_debug_stats stats[RTE_MAX_LCORE + 1]; #endif } __rte_cache_aligned; =20 @@ -302,10 +304,7 @@ struct rte_mempool { */ #ifdef RTE_LIBRTE_MEMPOOL_DEBUG #define RTE_MEMPOOL_STAT_ADD(mp, name, n) do { \ - unsigned __lcore_id =3D rte_lcore_id(); \ - if (__lcore_id < RTE_MAX_LCORE) { \ - mp->stats[__lcore_id].name +=3D n; \ - } \ + (mp)->stats[rte_lcore_id()].name +=3D n; } while (0) #else #define RTE_MEMPOOL_STAT_ADD(mp, name, n) do {} while (0)