From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74EDAA0A0C; Fri, 2 Jul 2021 08:20:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1D9C641392; Fri, 2 Jul 2021 08:19:02 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2047.outbound.protection.outlook.com [40.107.93.47]) by mails.dpdk.org (Postfix) with ESMTP id 71B934137F for ; Fri, 2 Jul 2021 08:18:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WbMl6I2fYwiiGj9dPN5dygzsDxznH2KyivjLS096BKEJZ//Jsjv4FaW2rIJwnJRfPN0a7XFfCKiNZLoqkErcYIbWpbWyMvtDPCLn/hkk5Vn+6IlG9RLmriVt1V7/xwVwTNW9N6TDyDL2MaNLRrY+PJKHbYTwIDjFUXDfFVBc75J208k4KHPl5hSfLg2JSEdyJUWoZ3ogJDYLmuFkpxERh0SNRWAVQHXJdF15woyDAmTgU/Z9g40GVeu4GL1htiD18Pat+E1JcaaIQTOfpgPRYqsXDVXdjb1/+v4CuXCLuXTocUgatnkW2r6qxJ1Zuz8J7f61sk+VcbhNJpoqifVakQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EkgzUpz5t/xcsSlUPDMfJQbwf+SjZKFxgQuaQST2xPs=; b=MKsd8171A8Y3/YRMK78wbTTOiiGvo/TfmZzkC+Kc3RO0kTVKtawh7gBElADu6PCc4pYZU3GplmwWre4hZ+25u+IJbtaWzXfaz+QO7ckpkdd0+ovtVXWsmVInGaKjzo/8zR/9Qj7cMryaMm6+KhY8ARhXfU+VE1Ej3r9kc8iJbI2rqt6qlVLXYYmnCPaIaiuDjC7KzmJtkOlX4tbPAqT+1OKG1aam/3eQ2FRghN2T2lgIcs3XfXLWGKrOkj67eaXGXv2JSS3uT9nhL1fnvaFlA2fO61jRmnlX7UkfIZGpPnlv0KhanNZRlh5om1Vqf/euuGj6lp5fXqB9gJa6KG4OqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EkgzUpz5t/xcsSlUPDMfJQbwf+SjZKFxgQuaQST2xPs=; b=JGNOB3FtZgve1l0D0dN/pM2m1l2IxpgbN0MZsiFG50/b+1FgO9FUJk1epRdzFbshpmno1BUVjjIfOVkg7mN9wddPhej4+LWAb4yTq1lN7ZZl1Hdc1W1os/fHS1twKJnKuL9GyBvmaecY8SIt2zoofxs6u7peY4rzhXy/01VSfDkSN4oJg0r/a9XutKRI8xgi0QCemlLgvS35Y1ecVsiPVNmZAKKc7AHqMKRhNwYwq9By2M5ROASIZo7SGIsPoLY3f5B4LA78SPPlJG4Vn2TPbDZtPM2bg6Uuxzvi1ELf6Np49K06eaNDNPI4dxjoUsASqXxQWZI7/4mdi3QdXc4yDA== Received: from BN6PR1201CA0009.namprd12.prod.outlook.com (2603:10b6:405:4c::19) by CH2PR12MB3797.namprd12.prod.outlook.com (2603:10b6:610:27::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Fri, 2 Jul 2021 06:18:58 +0000 Received: from BN8NAM11FT026.eop-nam11.prod.protection.outlook.com (2603:10b6:405:4c:cafe::d8) by BN6PR1201CA0009.outlook.office365.com (2603:10b6:405:4c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Fri, 2 Jul 2021 06:18:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT026.mail.protection.outlook.com (10.13.177.51) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Fri, 2 Jul 2021 06:18:57 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 2 Jul 2021 06:18:55 +0000 From: Suanming Mou To: , CC: , , Date: Fri, 2 Jul 2021 09:18:09 +0300 Message-ID: <20210702061816.10454-16-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210702061816.10454-1-suanmingm@nvidia.com> References: <20210527093403.1153127-1-suanmingm@nvidia.com> <20210702061816.10454-1-suanmingm@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1fd84974-efe3-45c1-4aa9-08d93d2146d4 X-MS-TrafficTypeDiagnostic: CH2PR12MB3797: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:71; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bTxq99QLYgbvRv7HHPh4GkrrcxgAMaLnwiOClzmBmCjpgRJBPQv2Ly4289RPC3wIqdvRo9tgReddL9rqffdaxdWub9BUe+y0qy4oUXfRZFewtVAFLVes7AS+zN4LP3K7vXMc69JqyV8Psf2ZxhTgBBbgk9/dKRADrgjDlc0oCwGd2miTXBEK48Q210/f0AhlROrexP9wld9I/hw31ItborVNkXAoHk7VlVfYR5XGhBtCbSvy5oO6Bn3qGdmAeJOcfsEHoJZiRl1yoyPkeWi0MadITC9N1AE5mSQ61a/GcrGKOeDJnEHTOuKN16FxKs2z7TTGzRe+KMcXfd4hBH/ao45Nv2ALf/tIcyvWr2oqGa9swYgc8p+34stdM0YEfrpHL764dKQXfhCJ6XZyC82fEeIw/8DBl/bURfKSpy+LpmA5DeT7rIQ6dQol90XuAE7Vtdr3gctbE20KiaMh4lxxJh/RN8l4OYv3hR2FruUucmTMlJDXsCHTEtHShGEBw6PIcTTb5YUwUUu9hiUlgGpQA1d5jgU2CKmlnNcbhWuCSOlTQDaATjft0Oo4nDMtd0JdUCqXh4v/zoDTwfO4RWFfLEMGDYX8S6Ga4d6E73Hw/ctht6JaEF1/v/tuDSgyuDxLyKFA03v14MzkWOpJJhygFVul6+6ch6lbEae9jMFsdsNAXI+xzzPmhYKkyqGWWdP6dP8NiGkMve2tb1p9v7NVbg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(396003)(136003)(39860400002)(346002)(46966006)(36840700001)(5660300002)(16526019)(186003)(36860700001)(47076005)(8936002)(356005)(7636003)(6636002)(2906002)(55016002)(83380400001)(6286002)(82740400003)(1076003)(86362001)(36756003)(70586007)(7696005)(426003)(478600001)(4326008)(336012)(2616005)(26005)(36906005)(70206006)(82310400003)(54906003)(110136005)(316002)(6666004)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2021 06:18:57.9289 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fd84974-efe3-45c1-4aa9-08d93d2146d4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT026.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3797 Subject: [dpdk-dev] [PATCH v3 15/22] common/mlx5: allocate cache list memory individually X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, the list's local cache instance memory is allocated with the list. As the local cache instance array size is RTE_MAX_LCORE, most of the cases the system will only have very limited cores. allocate the instance memory individually per core will be more economic to the memory. This commit changes the instance array to pointer array, allocate the local cache memory only when the core is to be used. Signed-off-by: Suanming Mou Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_utils.c | 62 ++++++++++++++++++------- drivers/common/mlx5/mlx5_common_utils.h | 2 +- 2 files changed, 45 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_utils.c b/drivers/common/mlx5/mlx5_common_utils.c index 4e385c616a..f75b1cb0da 100644 --- a/drivers/common/mlx5/mlx5_common_utils.c +++ b/drivers/common/mlx5/mlx5_common_utils.c @@ -15,14 +15,13 @@ static int mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, - bool lcores_share, mlx5_list_create_cb cb_create, + bool lcores_share, struct mlx5_list_cache *gc, + mlx5_list_create_cb cb_create, mlx5_list_match_cb cb_match, mlx5_list_remove_cb cb_remove, mlx5_list_clone_cb cb_clone, mlx5_list_clone_free_cb cb_clone_free) { - int i; - if (!cb_match || !cb_create || !cb_remove || !cb_clone || !cb_clone_free) { rte_errno = EINVAL; @@ -38,9 +37,11 @@ mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, list->cb_clone = cb_clone; list->cb_clone_free = cb_clone_free; rte_rwlock_init(&list->lock); + if (lcores_share) { + list->cache[RTE_MAX_LCORE] = gc; + LIST_INIT(&list->cache[RTE_MAX_LCORE]->h); + } DRV_LOG(DEBUG, "mlx5 list %s initialized.", list->name); - for (i = 0; i <= RTE_MAX_LCORE; i++) - LIST_INIT(&list->cache[i].h); return 0; } @@ -53,11 +54,16 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_list *list; + struct mlx5_list_cache *gc = NULL; - list = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*list), 0, SOCKET_ID_ANY); + list = mlx5_malloc(MLX5_MEM_ZERO, + sizeof(*list) + (lcores_share ? sizeof(*gc) : 0), + 0, SOCKET_ID_ANY); if (!list) return NULL; - if (mlx5_list_init(list, name, ctx, lcores_share, + if (lcores_share) + gc = (struct mlx5_list_cache *)(list + 1); + if (mlx5_list_init(list, name, ctx, lcores_share, gc, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(list); @@ -69,7 +75,8 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, static struct mlx5_list_entry * __list_lookup(struct mlx5_list *list, int lcore_index, void *ctx, bool reuse) { - struct mlx5_list_entry *entry = LIST_FIRST(&list->cache[lcore_index].h); + struct mlx5_list_entry *entry = + LIST_FIRST(&list->cache[lcore_index]->h); uint32_t ret; while (entry != NULL) { @@ -121,14 +128,14 @@ mlx5_list_cache_insert(struct mlx5_list *list, int lcore_index, lentry->ref_cnt = 1u; lentry->gentry = gentry; lentry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, lentry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, lentry, next); return lentry; } static void __list_cache_clean(struct mlx5_list *list, int lcore_index) { - struct mlx5_list_cache *c = &list->cache[lcore_index]; + struct mlx5_list_cache *c = list->cache[lcore_index]; struct mlx5_list_entry *entry = LIST_FIRST(&c->h); uint32_t inv_cnt = __atomic_exchange_n(&c->inv_cnt, 0, __ATOMIC_RELAXED); @@ -161,6 +168,17 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) rte_errno = ENOTSUP; return NULL; } + if (unlikely(!list->cache[lcore_index])) { + list->cache[lcore_index] = mlx5_malloc(0, + sizeof(struct mlx5_list_cache), + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (!list->cache[lcore_index]) { + rte_errno = ENOMEM; + return NULL; + } + list->cache[lcore_index]->inv_cnt = 0; + LIST_INIT(&list->cache[lcore_index]->h); + } /* 0. Free entries that was invalidated by other lcores. */ __list_cache_clean(list, lcore_index); /* 1. Lookup in local cache. */ @@ -186,7 +204,7 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) entry->ref_cnt = 1u; if (!list->lcores_share) { entry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "MLX5 list %s c%d entry %p new: %u.", list->name, lcore_index, (void *)entry, entry->ref_cnt); @@ -217,10 +235,10 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) } } /* 5. Update lists. */ - LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE].h, entry, next); + LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE]->h, entry, next); list->gen_cnt++; rte_rwlock_write_unlock(&list->lock); - LIST_INSERT_HEAD(&list->cache[lcore_index].h, local_entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, local_entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "mlx5 list %s entry %p new: %u.", list->name, (void *)entry, entry->ref_cnt); @@ -245,7 +263,7 @@ mlx5_list_unregister(struct mlx5_list *list, else list->cb_remove(list->ctx, entry); } else if (likely(lcore_idx != -1)) { - __atomic_add_fetch(&list->cache[entry->lcore_idx].inv_cnt, 1, + __atomic_add_fetch(&list->cache[entry->lcore_idx]->inv_cnt, 1, __ATOMIC_RELAXED); } else { return 0; @@ -280,8 +298,10 @@ mlx5_list_uninit(struct mlx5_list *list) MLX5_ASSERT(list); for (i = 0; i <= RTE_MAX_LCORE; i++) { - while (!LIST_EMPTY(&list->cache[i].h)) { - entry = LIST_FIRST(&list->cache[i].h); + if (!list->cache[i]) + continue; + while (!LIST_EMPTY(&list->cache[i]->h)) { + entry = LIST_FIRST(&list->cache[i]->h); LIST_REMOVE(entry, next); if (i == RTE_MAX_LCORE) { list->cb_remove(list->ctx, entry); @@ -292,6 +312,8 @@ mlx5_list_uninit(struct mlx5_list *list) list->cb_clone_free(list->ctx, entry); } } + if (i != RTE_MAX_LCORE) + mlx5_free(list->cache[i]); } } @@ -320,6 +342,7 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_hlist *h; + struct mlx5_list_cache *gc; uint32_t act_size; uint32_t alloc_size; uint32_t i; @@ -333,7 +356,9 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, act_size = size; } alloc_size = sizeof(struct mlx5_hlist) + - sizeof(struct mlx5_hlist_bucket) * act_size; + sizeof(struct mlx5_hlist_bucket) * act_size; + if (lcores_share) + alloc_size += sizeof(struct mlx5_list_cache) * act_size; /* Using zmalloc, then no need to initialize the heads. */ h = mlx5_malloc(MLX5_MEM_ZERO, alloc_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); @@ -345,8 +370,10 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, h->mask = act_size - 1; h->lcores_share = lcores_share; h->direct_key = direct_key; + gc = (struct mlx5_list_cache *)&h->buckets[act_size]; for (i = 0; i < act_size; i++) { if (mlx5_list_init(&h->buckets[i].l, name, ctx, lcores_share, + lcores_share ? &gc[i] : NULL, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(h); @@ -358,7 +385,6 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, return h; } - struct mlx5_list_entry * mlx5_hlist_lookup(struct mlx5_hlist *h, uint64_t key, void *ctx) { diff --git a/drivers/common/mlx5/mlx5_common_utils.h b/drivers/common/mlx5/mlx5_common_utils.h index 61b30a45ca..979dfafad4 100644 --- a/drivers/common/mlx5/mlx5_common_utils.h +++ b/drivers/common/mlx5/mlx5_common_utils.h @@ -104,7 +104,7 @@ struct mlx5_list { mlx5_list_remove_cb cb_remove; /**< entry remove callback. */ mlx5_list_clone_cb cb_clone; /**< entry clone callback. */ mlx5_list_clone_free_cb cb_clone_free; - struct mlx5_list_cache cache[RTE_MAX_LCORE + 1]; + struct mlx5_list_cache *cache[RTE_MAX_LCORE + 1]; /* Lcore cache, last index is the global cache. */ volatile uint32_t gen_cnt; /* List modification may update it. */ volatile uint32_t count; /* number of entries in list. */ -- 2.25.1