From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 36046A034F for ; Wed, 13 May 2020 17:27:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 19F731D641; Wed, 13 May 2020 17:27:54 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60064.outbound.protection.outlook.com [40.107.6.64]) by dpdk.org (Postfix) with ESMTP id 2196D1D620; Wed, 13 May 2020 17:27:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UvuM2Yg+Cntjer279Ib1wWrWrrfXyqVKoO5Do99ggGoKgbpARgkknFPDTmTLmA0+Y8Aw8YwhbFd6m0A4OyeK+ODeamsjdKQMMJYKaG8Zu5ZVuEapOqZ6gDLAEMEQAvjJTggXBKuEh8cloQa9Ym2EZOtSLO1ml/5Yb+klQJygaqtELNp880HdNAnyuEWlstK5W0y/8YxFjcs58DmO1JiLmUT4DWTxQCGdvkOjOPnUTJ4+8rNQTgvYsyj/TUkvdVADusqdBONWPCjJoTQaT/L/H3rgnYas87uZLV/5lpKehi4EaPHNguwj10YrMKdhBuwpMsZojV6XKAx5bNnMhPYUww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s52E+uOYmDp3/pHnudZvcdJ4kHEo1lZuU3/YTfXwy28=; b=auzbjd9BtKHV+NMI91ioPatFLdorBw4fUmxx+f2D0/N2zgSq2xCM9WFXw6qAZgG2sHB4CJW1Eql/6kmB0H91H4l99ewkUdKiwblPPM05fW6FjNCv915qUKxLgLC5nDJ0nfNYn80hj+SjmlbvpyfefvLYtfymuPSYKWIxvwhtqgTyfoV3Nd36AdGgcOMmubNDIrKu4gpJtCKi8SreIQE9KsXOVzDuhMoxgHc18n0jp2QfM3OKij/MsR2XEJoWEFYQaLI3SkpdY0Fj8XoGZdSdVd8XDXqFFMvlNjv1Y9lH8n+goIKxzfo6hO9s+/QjopExy1gQR6DgOvuL7DJxGh2M+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s52E+uOYmDp3/pHnudZvcdJ4kHEo1lZuU3/YTfXwy28=; b=ccIfeLTmVPAepPxnyTiEmeERPNbHx4dRmyIycdfaX1J8zbdeQOS34Cw3zQRI8uzxp8NiQdqG7w/Y1AxRXBAAluTsZI4lFUBp6WK+sJaqKewBAxsc9565FJnFIR13Pn2G7nlb1fHvTfGP6+aU7HsqRh5zstgePPf7wUUmFemhjSc= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (2603:10a6:205:8::26) by AM4PR05MB3348.eurprd05.prod.outlook.com (2603:10a6:205:5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28; Wed, 13 May 2020 15:27:49 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::bdf0:88a3:3a39:4be4]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::bdf0:88a3:3a39:4be4%7]) with mapi id 15.20.2979.033; Wed, 13 May 2020 15:27:49 +0000 From: Slava Ovsiienko To: Matan Azrad , "dev@dpdk.org" CC: "stable@dpdk.org" Thread-Topic: [PATCH] net/mlx5: fix counter container resize Thread-Index: AQHWKFxA7wzI4mCS5k6nge63Z0WLoaimJQzA Date: Wed, 13 May 2020 15:27:49 +0000 Message-ID: References: <1589287933-91969-1-git-send-email-matan@mellanox.com> In-Reply-To: <1589287933-91969-1-git-send-email-matan@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 3a8af19c-90c4-42ba-5cf8-08d7f7523218 x-ms-traffictypediagnostic: AM4PR05MB3348:|AM4PR05MB3348: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1284; x-forefront-prvs: 0402872DA1 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YvMdtHtC+CuqIdVw7DCR30ziim1haSHevkX5s4ixEfAxExGVjYISKk8qFVPUCsHK5F0qn1H3hiAuhCQDqIzX1OPuCV3Ux69whKxLLJol70+KHWUSD5VZmYRnsMa6U9zb4dmNf78MyghYOFu1G64fiPqHY+nJL0aWDajr4n1E1oyzIEgHEdXdz+FTtHDwqEtNSglSDoXYW1eUs8K8h9VB1P6UDMy9jxt2Z05PcGEyBxOubAAke7oQPuA1pvWTffPxutMBsiaHhQqaoabxm6tswFWwiypUjVo76/NSdHcf9irJpQ5PFIwGV52tz+V5g01VFpbT4bVBl3DVmvehy6u8zX3AtD0hqftrTDrFmGW0t3wk+zFssaURQLrXnLsVorRc8b5k/N1kPKF36m/Nc7ATY40EXwE5Ef5ej/OV/ARvVGtBcR6Kz5NOCVnsXm5qFzMekH9PaGOwaV8GUVJ+hZI/QY/hIu7bht4xdbxUcGKSkPtnNb+jA0jWBGTSEKl0sU9IWOvdLO5vxPNkLX4im3Kp4Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR05MB3265.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(366004)(136003)(39860400002)(376002)(346002)(33430700001)(478600001)(9686003)(55016002)(66946007)(450100002)(8936002)(66556008)(33656002)(76116006)(64756008)(2906002)(52536014)(86362001)(66476007)(66446008)(26005)(33440700001)(53546011)(6506007)(30864003)(4326008)(7696005)(110136005)(186003)(316002)(71200400001)(8676002)(5660300002)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: DjKI8FLkW8MZjbfW+z10Ek6DgM+jiqksssjZ+XN0Fq1Lkcxz6dwou3bGG+dwtUuiEFYUCc4/Ix5el7zpNXSDbJo81RYy/o0l6r2rPQn1CE54YTE9KvTDVGrsdWaV1V/NGFEjqCuUrD7aMedq+2YK6RNk+KVNnyPx6im3HAIDQwL1fWpcak+2s/BBsN7wNRxu/wZ+4bVoN771ovDPvEXp1x/THcWsUvkp+OEgNrnYDtFSlpi7gKqvHX3v4A6BVwwZJWDSBKYQzJB0OyPLMhvXA2EKpB8fZb1fUtdYXS+Ktt+zZezM2M0acxJddxvw4PE8bvaEUiD2kHTVz7C2stFKK9lhkVGLDGWxOq2VOuYojc2KDyTHKaEZbsSaeXuvI0cWFmyEKc3cdIFfqNaRo/3VLPsRUrZcq1+FkgfJ25LxLeG4jsm4lsnr0BF6YgzOQ/d1eCnUCklEs5WffSNE/rcSuAf3wNLPQWP/VAn6r76zjv4= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a8af19c-90c4-42ba-5cf8-08d7f7523218 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 May 2020 15:27:49.4851 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: L52+SvswXfKb89M2pJPi1nde9g0rk4E+1JhiBEeWPxaMYUYupM2cX/fHgBV5/lpte1f7LGltEJHoNmEQpnJ1ECaWeyB5mNfpRwqgC8Azxn0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3348 Subject: Re: [dpdk-stable] [PATCH] net/mlx5: fix counter container resize X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" > -----Original Message----- > From: Matan Azrad > Sent: Tuesday, May 12, 2020 15:52 > To: dev@dpdk.org > Cc: Slava Ovsiienko ; stable@dpdk.org > Subject: [PATCH] net/mlx5: fix counter container resize >=20 > The design of counter container resize used double buffer algorithm in or= der > to synchronize between the query thread to the control thread. > When the control thread detected resize need, it created new bigger buffe= r > for the counter pools in a new container and change the container index > atomically. > In case the query thread had not detect the previous resize before a new = one > need was detected by the control thread, the control thread returned > EAGAIN to the flow creation API used a COUNT action. >=20 > The rte_flow API doesn't allow unblocked commands and doesn't expect to > get EAGAIN error type. >=20 > So, when a lot of flows were created between 2 different periodic queries= , 2 > different resizes might try to be created and caused EAGAIN error. > This behavior may blame flow creations. >=20 > Change the synchronization way to use lock instead of double buffer > algorithm. >=20 > The critical section of this lock is very small, so flow insertion rate s= hould not > be decreased. >=20 > Fixes: ebbac312e448 ("net/mlx5: resize a full counter container") > Cc: stable@dpdk.org Acked-by: Viacheslav Ovsiienko >=20 > Signed-off-by: Matan Azrad > --- > drivers/net/mlx5/mlx5.c | 70 +++++++++++------------ > drivers/net/mlx5/mlx5.h | 23 +++++--- > drivers/net/mlx5/mlx5_flow.c | 37 ++++--------- > drivers/net/mlx5/mlx5_flow.h | 6 -- > drivers/net/mlx5/mlx5_flow_dv.c | 110 +++++++++++++++----------------= ---- > -- > drivers/net/mlx5/mlx5_flow_verbs.c | 6 +- > 6 files changed, 104 insertions(+), 148 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 4f704cb..1445809 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -467,13 +467,13 @@ struct mlx5_flow_id_pool * static void > mlx5_flow_counters_mng_init(struct mlx5_ibv_shared *sh) { > - uint8_t i, age; > + int i; >=20 > - sh->cmng.age =3D 0; > + memset(&sh->cmng, 0, sizeof(sh->cmng)); > TAILQ_INIT(&sh->cmng.flow_counters); > - for (age =3D 0; age < RTE_DIM(sh->cmng.ccont[0]); ++age) { > - for (i =3D 0; i < RTE_DIM(sh->cmng.ccont); ++i) > - TAILQ_INIT(&sh->cmng.ccont[i][age].pool_list); > + for (i =3D 0; i < MLX5_CCONT_TYPE_MAX; ++i) { > + TAILQ_INIT(&sh->cmng.ccont[i].pool_list); > + rte_spinlock_init(&sh->cmng.ccont[i].resize_sl); > } > } >=20 > @@ -504,7 +504,7 @@ struct mlx5_flow_id_pool * > mlx5_flow_counters_mng_close(struct mlx5_ibv_shared *sh) { > struct mlx5_counter_stats_mem_mng *mng; > - uint8_t i, age =3D 0; > + int i; > int j; > int retries =3D 1024; >=20 > @@ -515,42 +515,34 @@ struct mlx5_flow_id_pool * > break; > rte_pause(); > } > + for (i =3D 0; i < MLX5_CCONT_TYPE_MAX; ++i) { > + struct mlx5_flow_counter_pool *pool; > + uint32_t batch =3D !!(i > 1); >=20 > - for (age =3D 0; age < RTE_DIM(sh->cmng.ccont[0]); ++age) { > - for (i =3D 0; i < RTE_DIM(sh->cmng.ccont); ++i) { > - struct mlx5_flow_counter_pool *pool; > - uint32_t batch =3D !!(i % 2); > - > - if (!sh->cmng.ccont[i][age].pools) > - continue; > - pool =3D TAILQ_FIRST(&sh- > >cmng.ccont[i][age].pool_list); > - while (pool) { > - if (batch) { > - if (pool->min_dcs) > - claim_zero > - (mlx5_devx_cmd_destroy > - (pool->min_dcs)); > - } > - for (j =3D 0; j < MLX5_COUNTERS_PER_POOL; > ++j) { > - if (MLX5_POOL_GET_CNT(pool, j)- > >action) > - claim_zero > - (mlx5_glue- > >destroy_flow_action > - (MLX5_POOL_GET_CNT > - (pool, j)->action)); > - if (!batch && > MLX5_GET_POOL_CNT_EXT > - (pool, j)->dcs) > - > claim_zero(mlx5_devx_cmd_destroy > - > (MLX5_GET_POOL_CNT_EXT > - (pool, j)->dcs)); > - } > - TAILQ_REMOVE(&sh- > >cmng.ccont[i][age].pool_list, > - pool, next); > - rte_free(pool); > - pool =3D TAILQ_FIRST > - (&sh->cmng.ccont[i][age].pool_list); > + if (!sh->cmng.ccont[i].pools) > + continue; > + pool =3D TAILQ_FIRST(&sh->cmng.ccont[i].pool_list); > + while (pool) { > + if (batch && pool->min_dcs) > + claim_zero(mlx5_devx_cmd_destroy > + (pool->min_dcs)); > + for (j =3D 0; j < MLX5_COUNTERS_PER_POOL; ++j) { > + if (MLX5_POOL_GET_CNT(pool, j)->action) > + claim_zero > + (mlx5_glue->destroy_flow_action > + (MLX5_POOL_GET_CNT > + (pool, j)->action)); > + if (!batch && MLX5_GET_POOL_CNT_EXT > + (pool, j)->dcs) > + claim_zero(mlx5_devx_cmd_destroy > + > (MLX5_GET_POOL_CNT_EXT > + (pool, j)->dcs)); > } > - rte_free(sh->cmng.ccont[i][age].pools); > + TAILQ_REMOVE(&sh->cmng.ccont[i].pool_list, pool, > next); > + rte_free(pool); > + pool =3D TAILQ_FIRST(&sh->cmng.ccont[i].pool_list); > } > + rte_free(sh->cmng.ccont[i].pools); > } > mng =3D LIST_FIRST(&sh->cmng.mem_mngs); > while (mng) { > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > 1740d4a..d9f5d81 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -227,13 +227,11 @@ struct mlx5_drop { #define CNTEXT_SIZE > (sizeof(struct mlx5_flow_counter_ext)) #define AGE_SIZE (sizeof(struct > mlx5_age_param)) > #define MLX5_AGING_TIME_DELAY 7 > - > #define CNT_POOL_TYPE_EXT (1 << 0) > #define CNT_POOL_TYPE_AGE (1 << 1) > #define IS_EXT_POOL(pool) (((pool)->type) & CNT_POOL_TYPE_EXT) > #define IS_AGE_POOL(pool) (((pool)->type) & CNT_POOL_TYPE_AGE) > #define MLX_CNT_IS_AGE(counter) ((counter) & MLX5_CNT_AGE_OFFSET ? 1 > : 0) > - > #define MLX5_CNT_LEN(pool) \ > (CNT_SIZE + \ > (IS_AGE_POOL(pool) ? AGE_SIZE : 0) + \ @@ -270,6 +268,17 @@ > enum { > AGE_TMOUT, /* Timeout, wait for rte_flow_get_aged_flows and > destroy. */ }; >=20 > +#define MLX5_CNT_CONTAINER(sh, batch, age) (&(sh)->cmng.ccont \ > + [(batch) * 2 + (age)]) > + > +enum { > + MLX5_CCONT_TYPE_SINGLE, > + MLX5_CCONT_TYPE_SINGLE_FOR_AGE, > + MLX5_CCONT_TYPE_BATCH, > + MLX5_CCONT_TYPE_BATCH_FOR_AGE, > + MLX5_CCONT_TYPE_MAX, > +}; > + > /* Counter age parameter. */ > struct mlx5_age_param { > rte_atomic16_t state; /**< Age state. */ @@ -313,7 +322,6 @@ > struct mlx5_flow_counter_ext { > }; > }; >=20 > - > TAILQ_HEAD(mlx5_counters, mlx5_flow_counter); >=20 > /* Generic counter pool structure - query is in pool resolution. */ @@ - > 358,17 +366,16 @@ struct mlx5_counter_stats_raw { struct > mlx5_pools_container { > rte_atomic16_t n_valid; /* Number of valid pools. */ > uint16_t n; /* Number of pools. */ > + rte_spinlock_t resize_sl; /* The resize lock. */ > struct mlx5_counter_pools pool_list; /* Counter pool list. */ > struct mlx5_flow_counter_pool **pools; /* Counter pool array. */ > - struct mlx5_counter_stats_mem_mng *init_mem_mng; > + struct mlx5_counter_stats_mem_mng *mem_mng; > /* Hold the memory management for the next allocated pools raws. > */ }; >=20 > /* Counter global management structure. */ struct > mlx5_flow_counter_mng { > - uint8_t mhi[2][2]; /* master \ host and age \ no age container index. > */ > - struct mlx5_pools_container ccont[2 * 2][2]; > - /* master \ host and age \ no age pools container. */ > + struct mlx5_pools_container ccont[MLX5_CCONT_TYPE_MAX]; > struct mlx5_counters flow_counters; /* Legacy flow counter list. */ > uint8_t pending_queries; > uint8_t batch; > @@ -378,6 +385,7 @@ struct mlx5_flow_counter_mng { > LIST_HEAD(mem_mngs, mlx5_counter_stats_mem_mng) > mem_mngs; > LIST_HEAD(stat_raws, mlx5_counter_stats_raw) free_stat_raws; }; > + > #define MLX5_AGE_EVENT_NEW 1 > #define MLX5_AGE_TRIGGER 2 > #define MLX5_AGE_SET(age_info, BIT) \ > @@ -393,6 +401,7 @@ struct mlx5_age_info { > struct mlx5_counters aged_counters; /* Aged flow counter list. */ > rte_spinlock_t aged_sl; /* Aged flow counter list lock. */ }; > + > /* Per port data of shared IB device. */ struct mlx5_ibv_shared_port { > uint32_t ih_port_id; > diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c > index 08c7cdf..ae478a5 100644 > --- a/drivers/net/mlx5/mlx5_flow.c > +++ b/drivers/net/mlx5/mlx5_flow.c > @@ -5801,16 +5801,11 @@ struct mlx5_meter_domains_infos * static > uint32_t mlx5_get_all_valid_pool_count(struct mlx5_ibv_shared *sh) { > - uint8_t age, i; > + int i; > uint32_t pools_n =3D 0; > - struct mlx5_pools_container *cont; >=20 > - for (age =3D 0; age < RTE_DIM(sh->cmng.ccont[0]); ++age) { > - for (i =3D 0; i < 2 ; ++i) { > - cont =3D MLX5_CNT_CONTAINER(sh, i, 0, age); > - pools_n +=3D rte_atomic16_read(&cont->n_valid); > - } > - } > + for (i =3D 0; i < MLX5_CCONT_TYPE_MAX; ++i) > + pools_n +=3D rte_atomic16_read(&sh->cmng.ccont[i].n_valid); > return pools_n; > } >=20 > @@ -5855,32 +5850,19 @@ struct mlx5_meter_domains_infos * > uint8_t age =3D sh->cmng.age; > uint16_t pool_index =3D sh->cmng.pool_index; > struct mlx5_pools_container *cont; > - struct mlx5_pools_container *mcont; > struct mlx5_flow_counter_pool *pool; > + int cont_loop =3D MLX5_CCONT_TYPE_MAX; >=20 > if (sh->cmng.pending_queries >=3D MLX5_MAX_PENDING_QUERIES) > goto set_alarm; > next_container: > - cont =3D MLX5_CNT_CONTAINER(sh, batch, 1, age); > - mcont =3D MLX5_CNT_CONTAINER(sh, batch, 0, age); > - /* Check if resize was done and need to flip a container. */ > - if (cont !=3D mcont) { > - if (cont->pools) { > - /* Clean the old container. */ > - rte_free(cont->pools); > - memset(cont, 0, sizeof(*cont)); > - } > - rte_cio_wmb(); > - /* Flip the host container. */ > - sh->cmng.mhi[batch][age] ^=3D (uint8_t)2; > - cont =3D mcont; > - } > + cont =3D MLX5_CNT_CONTAINER(sh, batch, age); > + rte_spinlock_lock(&cont->resize_sl); > if (!cont->pools) { > - /* 2 empty containers case is unexpected. */ > - if (unlikely(batch !=3D sh->cmng.batch) && > - unlikely(age !=3D sh->cmng.age)) { > + rte_spinlock_unlock(&cont->resize_sl); > + /* Check if all the containers are empty. */ > + if (unlikely(--cont_loop =3D=3D 0)) > goto set_alarm; > - } > batch ^=3D 0x1; > pool_index =3D 0; > if (batch =3D=3D 0 && pool_index =3D=3D 0) { > @@ -5891,6 +5873,7 @@ struct mlx5_meter_domains_infos * > goto next_container; > } > pool =3D cont->pools[pool_index]; > + rte_spinlock_unlock(&cont->resize_sl); > if (pool->raw_hw) > /* There is a pool query in progress. */ > goto set_alarm; > diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h > index 7f5e01f..2c96677 100644 > --- a/drivers/net/mlx5/mlx5_flow.h > +++ b/drivers/net/mlx5/mlx5_flow.h > @@ -900,12 +900,6 @@ struct mlx5_flow_driver_ops { > mlx5_flow_get_aged_flows_t get_aged_flows; }; >=20 > - > -#define MLX5_CNT_CONTAINER(sh, batch, thread, age) (&(sh)->cmng.ccont > \ > - [(((sh)->cmng.mhi[batch][age] >> (thread)) & 0x1) * 2 + (batch)][age]) > -#define MLX5_CNT_CONTAINER_UNUSED(sh, batch, thread, age) (&(sh)- > >cmng.ccont \ > - [(~((sh)->cmng.mhi[batch][age] >> (thread)) & 0x1) * 2 + > (batch)][age]) > - > /* mlx5_flow.c */ >=20 > struct mlx5_flow_id_pool *mlx5_flow_id_pool_alloc(uint32_t max_id); diff= - > -git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c > index 4ebb7ce..c46a589 100644 > --- a/drivers/net/mlx5/mlx5_flow_dv.c > +++ b/drivers/net/mlx5/mlx5_flow_dv.c > @@ -4038,7 +4038,7 @@ struct field_modify_info modify_tcp[] =3D { > idx -=3D MLX5_CNT_BATCH_OFFSET; > batch =3D 1; > } > - cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, 0, age); > + cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, age); > MLX5_ASSERT(idx / MLX5_COUNTERS_PER_POOL < cont->n); > pool =3D cont->pools[idx / MLX5_COUNTERS_PER_POOL]; > MLX5_ASSERT(pool); > @@ -4162,69 +4162,55 @@ struct field_modify_info modify_tcp[] =3D { > * Whether the pool is for Aging counter. > * > * @return > - * The new container pointer on success, otherwise NULL and rte_errno = is > set. > + * 0 on success, otherwise negative errno value and rte_errno is set. > */ > -static struct mlx5_pools_container * > +static int > flow_dv_container_resize(struct rte_eth_dev *dev, > uint32_t batch, uint32_t age) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > - struct mlx5_pools_container *cont =3D > - MLX5_CNT_CONTAINER(priv->sh, batch, 0, age); > - struct mlx5_pools_container *new_cont =3D > - MLX5_CNT_CONTAINER_UNUSED(priv->sh, batch, 0, > age); > + struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, batch, > + age); > struct mlx5_counter_stats_mem_mng *mem_mng =3D NULL; > + void *old_pools =3D cont->pools; > uint32_t resize =3D cont->n + MLX5_CNT_CONTAINER_RESIZE; > uint32_t mem_size =3D sizeof(struct mlx5_flow_counter_pool *) * > resize; > - int i; > + void *pools =3D rte_calloc(__func__, 1, mem_size, 0); >=20 > - /* Fallback mode has no background thread. Skip the check. */ > - if (!priv->counter_fallback && > - cont !=3D MLX5_CNT_CONTAINER(priv->sh, batch, 1, age)) { > - /* The last resize still hasn't detected by the host thread. */ > - rte_errno =3D EAGAIN; > - return NULL; > - } > - new_cont->pools =3D rte_calloc(__func__, 1, mem_size, 0); > - if (!new_cont->pools) { > + if (!pools) { > rte_errno =3D ENOMEM; > - return NULL; > + return -ENOMEM; > } > - if (cont->n) > - memcpy(new_cont->pools, cont->pools, cont->n * > - sizeof(struct mlx5_flow_counter_pool *)); > + if (old_pools) > + memcpy(pools, old_pools, cont->n * > + sizeof(struct mlx5_flow_counter_pool > *)); > /* > * Fallback mode query the counter directly, no background query > * resources are needed. > */ > if (!priv->counter_fallback) { > + int i; > + > mem_mng =3D flow_dv_create_counter_stat_mem_mng(dev, > - MLX5_CNT_CONTAINER_RESIZE + > MLX5_MAX_PENDING_QUERIES); > + MLX5_CNT_CONTAINER_RESIZE + > MLX5_MAX_PENDING_QUERIES); > if (!mem_mng) { > - rte_free(new_cont->pools); > - return NULL; > + rte_free(pools); > + return -ENOMEM; > } > for (i =3D 0; i < MLX5_MAX_PENDING_QUERIES; ++i) > LIST_INSERT_HEAD(&priv->sh- > >cmng.free_stat_raws, > mem_mng->raws + > MLX5_CNT_CONTAINER_RESIZE + > i, next); > - } else { > - /* > - * Release the old container pools directly as no background > - * thread helps that. > - */ > - rte_free(cont->pools); > } > - new_cont->n =3D resize; > - rte_atomic16_set(&new_cont->n_valid, rte_atomic16_read(&cont- > >n_valid)); > - TAILQ_INIT(&new_cont->pool_list); > - TAILQ_CONCAT(&new_cont->pool_list, &cont->pool_list, next); > - new_cont->init_mem_mng =3D mem_mng; > - rte_cio_wmb(); > - /* Flip the master container. */ > - priv->sh->cmng.mhi[batch][age] ^=3D (uint8_t)1; > - return new_cont; > + rte_spinlock_lock(&cont->resize_sl); > + cont->n =3D resize; > + cont->mem_mng =3D mem_mng; > + cont->pools =3D pools; > + rte_spinlock_unlock(&cont->resize_sl); > + if (old_pools) > + rte_free(old_pools); > + return 0; > } >=20 > /** > @@ -4296,22 +4282,19 @@ struct field_modify_info modify_tcp[] =3D { > * @return > * The pool container pointer on success, NULL otherwise and rte_errno= is > set. > */ > -static struct mlx5_pools_container * > +static struct mlx5_flow_counter_pool * > flow_dv_pool_create(struct rte_eth_dev *dev, struct mlx5_devx_obj *dcs, > uint32_t batch, uint32_t age) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > struct mlx5_flow_counter_pool *pool; > struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, batch, > - 0, age); > + age); > int16_t n_valid =3D rte_atomic16_read(&cont->n_valid); > uint32_t size =3D sizeof(*pool); >=20 > - if (cont->n =3D=3D n_valid) { > - cont =3D flow_dv_container_resize(dev, batch, age); > - if (!cont) > - return NULL; > - } > + if (cont->n =3D=3D n_valid && flow_dv_container_resize(dev, batch, age)= ) > + return NULL; > size +=3D MLX5_COUNTERS_PER_POOL * CNT_SIZE; > size +=3D (batch ? 0 : MLX5_COUNTERS_PER_POOL * CNTEXT_SIZE); > size +=3D (!age ? 0 : MLX5_COUNTERS_PER_POOL * AGE_SIZE); @@ - > 4322,8 +4305,8 @@ struct field_modify_info modify_tcp[] =3D { > } > pool->min_dcs =3D dcs; > if (!priv->counter_fallback) > - pool->raw =3D cont->init_mem_mng->raws + n_valid % > - > MLX5_CNT_CONTAINER_RESIZE; > + pool->raw =3D cont->mem_mng->raws + n_valid % > + > MLX5_CNT_CONTAINER_RESIZE; > pool->raw_hw =3D NULL; > pool->type =3D 0; > pool->type |=3D (batch ? 0 : CNT_POOL_TYPE_EXT); @@ -4351,7 > +4334,7 @@ struct field_modify_info modify_tcp[] =3D { > /* Pool initialization must be updated before host thread access. */ > rte_cio_wmb(); > rte_atomic16_add(&cont->n_valid, 1); > - return cont; > + return pool; > } >=20 > /** > @@ -4375,7 +4358,7 @@ struct field_modify_info modify_tcp[] =3D { > struct mlx5_flow_counter_pool *other; > struct mlx5_pools_container *cont; >=20 > - cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, 0, (age ^ 0x1)); > + cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, (age ^ 0x1)); > other =3D flow_dv_find_pool_by_id(cont, pool->min_dcs->id); > if (!other) > return; > @@ -4400,10 +4383,10 @@ struct field_modify_info modify_tcp[] =3D { > * Whether the pool is for counter that was allocated for aging. > * > * @return > - * The counter container pointer and @p cnt_free is set on success, > + * The counter pool pointer and @p cnt_free is set on success, > * NULL otherwise and rte_errno is set. > */ > -static struct mlx5_pools_container * > +static struct mlx5_flow_counter_pool * > flow_dv_counter_pool_prepare(struct rte_eth_dev *dev, > struct mlx5_flow_counter **cnt_free, > uint32_t batch, uint32_t age) > @@ -4415,7 +4398,7 @@ struct field_modify_info modify_tcp[] =3D { > struct mlx5_flow_counter *cnt; > uint32_t i; >=20 > - cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, 0, age); > + cont =3D MLX5_CNT_CONTAINER(priv->sh, batch, age); > if (!batch) { > /* bulk_bitmap must be 0 for single counter allocation. */ > dcs =3D mlx5_devx_cmd_flow_counter_alloc(priv->sh->ctx, 0); > @@ -4423,12 +4406,11 @@ struct field_modify_info modify_tcp[] =3D { > return NULL; > pool =3D flow_dv_find_pool_by_id(cont, dcs->id); > if (!pool) { > - cont =3D flow_dv_pool_create(dev, dcs, batch, age); > - if (!cont) { > + pool =3D flow_dv_pool_create(dev, dcs, batch, age); > + if (!pool) { > mlx5_devx_cmd_destroy(dcs); > return NULL; > } > - pool =3D TAILQ_FIRST(&cont->pool_list); > } else if (dcs->id < pool->min_dcs->id) { > rte_atomic64_set(&pool->a64_dcs, > (int64_t)(uintptr_t)dcs); > @@ -4440,7 +4422,7 @@ struct field_modify_info modify_tcp[] =3D { > TAILQ_INSERT_HEAD(&pool->counters, cnt, next); > MLX5_GET_POOL_CNT_EXT(pool, i)->dcs =3D dcs; > *cnt_free =3D cnt; > - return cont; > + return pool; > } > /* bulk_bitmap is in 128 counters units. */ > if (priv->config.hca_attr.flow_counter_bulk_alloc_bitmap & 0x4) > @@ -4449,18 +4431,17 @@ struct field_modify_info modify_tcp[] =3D { > rte_errno =3D ENODATA; > return NULL; > } > - cont =3D flow_dv_pool_create(dev, dcs, batch, age); > - if (!cont) { > + pool =3D flow_dv_pool_create(dev, dcs, batch, age); > + if (!pool) { > mlx5_devx_cmd_destroy(dcs); > return NULL; > } > - pool =3D TAILQ_FIRST(&cont->pool_list); > for (i =3D 0; i < MLX5_COUNTERS_PER_POOL; ++i) { > cnt =3D MLX5_POOL_GET_CNT(pool, i); > TAILQ_INSERT_HEAD(&pool->counters, cnt, next); > } > *cnt_free =3D MLX5_POOL_GET_CNT(pool, 0); > - return cont; > + return pool; > } >=20 > /** > @@ -4534,7 +4515,7 @@ struct field_modify_info modify_tcp[] =3D { > */ > uint32_t batch =3D (group && !shared && !priv->counter_fallback) ? 1 : > 0; > struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, batch, > - 0, age); > + age); > uint32_t cnt_idx; >=20 > if (!priv->config.devx) { > @@ -4573,10 +4554,9 @@ struct field_modify_info modify_tcp[] =3D { > cnt_free =3D NULL; > } > if (!cnt_free) { > - cont =3D flow_dv_counter_pool_prepare(dev, &cnt_free, > batch, age); > - if (!cont) > + pool =3D flow_dv_counter_pool_prepare(dev, &cnt_free, > batch, age); > + if (!pool) > return 0; > - pool =3D TAILQ_FIRST(&cont->pool_list); > } > if (!batch) > cnt_ext =3D MLX5_CNT_TO_CNT_EXT(pool, cnt_free); diff --git > a/drivers/net/mlx5/mlx5_flow_verbs.c > b/drivers/net/mlx5/mlx5_flow_verbs.c > index c403f72..01eec31 100644 > --- a/drivers/net/mlx5/mlx5_flow_verbs.c > +++ b/drivers/net/mlx5/mlx5_flow_verbs.c > @@ -56,8 +56,7 @@ > struct mlx5_flow_counter_pool **ppool) { > struct mlx5_priv *priv =3D dev->data->dev_private; > - struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, 0, 0, > - 0); > + struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, 0, > +0); > struct mlx5_flow_counter_pool *pool; >=20 > idx--; > @@ -152,8 +151,7 @@ > flow_verbs_counter_new(struct rte_eth_dev *dev, uint32_t shared, uint32_= t > id) { > struct mlx5_priv *priv =3D dev->data->dev_private; > - struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, 0, 0, > - 0); > + struct mlx5_pools_container *cont =3D MLX5_CNT_CONTAINER(priv- > >sh, 0, > +0); > struct mlx5_flow_counter_pool *pool =3D NULL; > struct mlx5_flow_counter_ext *cnt_ext =3D NULL; > struct mlx5_flow_counter *cnt =3D NULL; > -- > 1.8.3.1