From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id AC3E1A0096 for ; Mon, 8 Apr 2019 07:48:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7FF082C54; Mon, 8 Apr 2019 07:48:35 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80075.outbound.protection.outlook.com [40.107.8.75]) by dpdk.org (Postfix) with ESMTP id 58B662BD8 for ; Mon, 8 Apr 2019 07:48:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kJ6QLSMCwlbI8ZONRfQ6DyZvZALMy1b2IRzakAmH5cc=; b=MdYbVesnfsljI51S32Wrb1D/PbEhCEgxWwfj/vuKHprAr7JQlKUjbpgaryjwDnRJSpwQy8gtP1KbwwLhS79XnD5hhhHZUI46MfNA1Ler96Gzrt7x+7fHx29QdKG1jpbIbQEr9gOgDOMnhBGfAafCQe4duK+UV5kxnekNvj4Ti2E= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3876.eurprd05.prod.outlook.com (52.133.48.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1771.21; Mon, 8 Apr 2019 05:48:32 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::4192:b468:41e1:c323]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::4192:b468:41e1:c323%4]) with mapi id 15.20.1771.011; Mon, 8 Apr 2019 05:48:32 +0000 From: Shahaf Shuler To: Yongseok Koh CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v3 3/4] net/mlx5: remove device register remap Thread-Index: AQHU60+89UhckMtnYU+tSQi7Kgnfk6YxxECw Date: Mon, 8 Apr 2019 05:48:32 +0000 Message-ID: References: <20190325193627.19726-1-yskoh@mellanox.com> <20190405013357.14503-1-yskoh@mellanox.com> <20190405013357.14503-4-yskoh@mellanox.com> In-Reply-To: <20190405013357.14503-4-yskoh@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e7547a13-e066-48c1-1bee-08d6bbe5d5ca x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600139)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM0PR0502MB3876; x-ms-traffictypediagnostic: AM0PR0502MB3876: x-microsoft-antispam-prvs: x-forefront-prvs: 0001227049 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(136003)(346002)(396003)(376002)(39860400002)(199004)(189003)(71190400001)(76176011)(6436002)(14454004)(81166006)(81156014)(9686003)(8676002)(229853002)(68736007)(105586002)(53946003)(106356001)(6116002)(71200400001)(33656002)(3846002)(478600001)(6506007)(30864003)(25786009)(102836004)(86362001)(7696005)(476003)(6862004)(486006)(26005)(74316002)(97736004)(2906002)(8936002)(316002)(4326008)(446003)(11346002)(186003)(7736002)(5660300002)(14444005)(5024004)(256004)(99286004)(305945005)(66066001)(53936002)(6246003)(52536014)(55016002)(6636002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3876; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: inad015rUJ89cKsfxm6DXlcZC4qV76OVc2H8AtdK7J88yFcI9A67idQIuMCZYwp8tVQzOzS8j0hW+AhHDJmASpKRiVWIZHfteo1V1PBCGqNniE8BMR23oaTTvbGGgoI8F1jxzwUdalWggtmMzsY4HfAwhVzvzqBJO3qtGyffxj/vZ9NHgkuBlcIzh9W4hrnorltzQ3OtlX6hr2Qqzx9ciMLfCDXWiiq/NN4hyV/knC83cND2mDwvdcZ8mNL6t8HjbUGe/fFe/RahUQUX4+mkG7Y9cv8oSgVeNG8ap83wjKIVkJjq+dRGKfBxL+KdDVH3w+Fc09Px53iuhZMJMuEp3eZnE0+LOJCo2f7jqzCthLhX+zeYEqW2vBeeqHSfi+M78y87T9gppUNbGCMNiZjLhVGM6jhmz5/2TSQzacyK0HM= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7547a13-e066-48c1-1bee-08d6bbe5d5ca X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Apr 2019 05:48:32.7040 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3876 Subject: Re: [dpdk-dev] [PATCH v3 3/4] net/mlx5: remove device register remap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190408054832.dpvHb-bsMhdttHG5T141ys9Arj1Fx7tqy-UWe1LN2Zs@z> Hi Koh, See small comments below. Same for mlx4 patch. Friday, April 5, 2019 4:34 AM, Yongseok Koh: > Subject: [dpdk-dev] [PATCH v3 3/4] net/mlx5: remove device register remap >=20 > UAR (User Access Region) register does not need to be remapped for > primary process but it should be remapped only for secondary process. UAR > register table is in the process private structure in rte_eth_devices[], > (struct mlx5_proc_priv *)rte_eth_devices[port_id].process_private >=20 > The actual UAR table follows the data structure and the table is used for= both > Tx and Rx. >=20 > For Tx, BlueFlame in UAR is used to ring the doorbell. MLX5_TX_BFREG(txq) > is defined to get a register for the txq. Processes access its own privat= e data > to acquire the register from the UAR table. >=20 > For Rx, the doorbell in UAR is required in arming CQ event. However, it i= s a > known issue that the register isn't remapped for secondary process. >=20 > Signed-off-by: Yongseok Koh > --- > drivers/net/mlx5/mlx5.c | 198 +++++-----------------------------= ------ > drivers/net/mlx5/mlx5.h | 15 ++- > drivers/net/mlx5/mlx5_ethdev.c | 17 ++++ > drivers/net/mlx5/mlx5_rxtx.h | 11 ++- > drivers/net/mlx5/mlx5_trigger.c | 6 -- > drivers/net/mlx5/mlx5_txq.c | 180 ++++++++++++++++++++++------------= - > - [...] > /** > @@ -1182,12 +1010,32 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > } > DRV_LOG(DEBUG, "naming Ethernet device \"%s\"", name); > if (rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY) { > + struct mlx5_proc_priv *ppriv; > + size_t ppriv_size; > + > eth_dev =3D rte_eth_dev_attach_secondary(name); > if (eth_dev =3D=3D NULL) { > DRV_LOG(ERR, "can not attach rte ethdev"); > rte_errno =3D ENOMEM; > return NULL; > } > + priv =3D eth_dev->data->dev_private; > + /* > + * UAR register table follows the process private structure. > + * BlueFlame registers for Tx queues come first and registers > + * for Rx queues follows. > + */ > + ppriv_size =3D sizeof(struct mlx5_proc_priv) + > + (priv->rxqs_n + priv->txqs_n) * sizeof(void *); Why you add also the rxqs_n? why not only the txqs?=20 > + ppriv =3D rte_malloc_socket("mlx5_proc_priv", ppriv_size, > + RTE_CACHE_LINE_SIZE, > + dpdk_dev->numa_node); > + if (!ppriv) { > + rte_errno =3D ENOMEM; > + return NULL; > + } > + ppriv->uar_table_sz =3D ppriv_size; > + eth_dev->process_private =3D ppriv; > eth_dev->device =3D dpdk_dev; > eth_dev->dev_ops =3D &mlx5_dev_sec_ops; > /* Receive command fd from primary process */ @@ -1195,7 > +1043,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > if (err < 0) > return NULL; > /* Remap UAR for Tx queues. */ > - err =3D mlx5_tx_uar_remap(eth_dev, err); > + err =3D mlx5_tx_uar_init_secondary(eth_dev, err); > if (err) > return NULL; > /* > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > 699c8fcf6d..1ac4ad71b1 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -97,8 +97,6 @@ struct mlx5_shared_data { > /* Global spinlock for primary and secondary processes. */ > int init_done; /* Whether primary has done initialization. */ > unsigned int secondary_cnt; /* Number of secondary processes > init'd. */ > - void *uar_base; > - /* Reserved UAR address space for TXQ UAR(hw doorbell) mapping. > */ > struct mlx5_dev_list mem_event_cb_list; > rte_rwlock_t mem_event_rwlock; > }; > @@ -106,8 +104,6 @@ struct mlx5_shared_data { > /* Per-process data structure, not visible to other processes. */ struc= t > mlx5_local_data { > int init_done; /* Whether a secondary has done initialization. */ > - void *uar_base; > - /* Reserved UAR address space for TXQ UAR(hw doorbell) mapping. > */ > }; >=20 > extern struct mlx5_shared_data *mlx5_shared_data; @@ -282,6 +278,17 > @@ struct mlx5_ibv_shared { > struct mlx5_ibv_shared_port port[]; /* per device port data array. */ > }; >=20 > +/* Per-process private structure. */ > +struct mlx5_proc_priv { > + size_t uar_table_sz; > + /* Size of UAR register table. */ > + void *uar_table[]; > + /* Table of UAR registers for each process. */ }; > + > +#define MLX5_PROC_PRIV(port_id) \ > + ((struct mlx5_proc_priv *)rte_eth_devices[port_id].process_private) > + > struct mlx5_priv { > LIST_ENTRY(mlx5_priv) mem_event_cb; > /**< Called by memory event callback. */ diff --git > a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index > 9ae9dddd3c..42297f11c9 100644 > --- a/drivers/net/mlx5/mlx5_ethdev.c > +++ b/drivers/net/mlx5/mlx5_ethdev.c > @@ -382,6 +382,8 @@ int > mlx5_dev_configure(struct rte_eth_dev *dev) { > struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_proc_priv *ppriv; > + size_t ppriv_size; > unsigned int rxqs_n =3D dev->data->nb_rx_queues; > unsigned int txqs_n =3D dev->data->nb_tx_queues; > unsigned int i; > @@ -450,6 +452,21 @@ mlx5_dev_configure(struct rte_eth_dev *dev) > if (++j =3D=3D rxqs_n) > j =3D 0; > } > + /* > + * UAR register table follows the process private structure. BlueFlame > + * registers for Tx queues come first and registers for Rx queues > + * follows. > + */ > + ppriv_size =3D sizeof(struct mlx5_proc_priv) + > + (priv->rxqs_n + priv->txqs_n) * sizeof(void *); Ditto.=20 > + ppriv =3D rte_malloc_socket("mlx5_proc_priv", ppriv_size, > + RTE_CACHE_LINE_SIZE, dev->device- > >numa_node); > + if (!ppriv) { > + rte_errno =3D ENOMEM; > + return -rte_errno; > + } > + ppriv->uar_table_sz =3D ppriv_size; > + dev->process_private =3D ppriv; > return 0; > } >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h > index 7b58063ceb..5d49892429 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.h > +++ b/drivers/net/mlx5/mlx5_rxtx.h > @@ -201,8 +201,8 @@ struct mlx5_txq_data { > volatile void *wqes; /* Work queue (use volatile to write into). */ > volatile uint32_t *qp_db; /* Work queue doorbell. */ > volatile uint32_t *cq_db; /* Completion queue doorbell. */ > - volatile void *bf_reg; /* Blueflame register remapped. */ > struct rte_mbuf *(*elts)[]; /* TX elements. */ > + uint16_t port_id; /* Port ID of device. */ > uint16_t idx; /* Queue index. */ > struct mlx5_txq_stats stats; /* TX queue counters. */ #ifndef > RTE_ARCH_64 @@ -231,9 +231,12 @@ struct mlx5_txq_ctrl { > struct mlx5_txq_ibv *ibv; /* Verbs queue object. */ > struct mlx5_priv *priv; /* Back pointer to private data. */ > off_t uar_mmap_offset; /* UAR mmap offset for non-primary > process. */ > - volatile void *bf_reg_orig; /* Blueflame register from verbs. */ > + void *bf_reg; /* BlueFlame register from Verbs. */ I guess you keep this one in order to get the VA offset for the secondary m= apping, right? Because otherwise we can take the bf_reg from the UAR table = on the process private. If so, better to rename it to uar_page_offset (or other name you like) in o= rder to avoid fields duplication.=20 > }; >=20 > +#define MLX5_TX_BFREG(txq) \ > + (MLX5_PROC_PRIV((txq)->port_id)->uar_table[(txq)->idx]) > + > /* mlx5_rxq.c */ >=20 > extern uint8_t rss_hash_default_key[]; > @@ -301,7 +304,7 @@ uint64_t mlx5_get_rx_queue_offloads(struct > rte_eth_dev *dev); int mlx5_tx_queue_setup(struct rte_eth_dev *dev, > uint16_t idx, uint16_t desc, > unsigned int socket, const struct rte_eth_txconf > *conf); void mlx5_tx_queue_release(void *dpdk_txq); -int > mlx5_tx_uar_remap(struct rte_eth_dev *dev, int fd); > +int mlx5_tx_uar_init_secondary(struct rte_eth_dev *dev, int fd); > struct mlx5_txq_ibv *mlx5_txq_ibv_new(struct rte_eth_dev *dev, uint16_t > idx); struct mlx5_txq_ibv *mlx5_txq_ibv_get(struct rte_eth_dev *dev, > uint16_t idx); int mlx5_txq_ibv_release(struct mlx5_txq_ibv *txq_ibv); @= @ > -704,7 +707,7 @@ static __rte_always_inline void > mlx5_tx_dbrec_cond_wmb(struct mlx5_txq_data *txq, volatile struct > mlx5_wqe *wqe, > int cond) > { > - uint64_t *dst =3D (uint64_t *)((uintptr_t)txq->bf_reg); > + uint64_t *dst =3D MLX5_TX_BFREG(txq); I guess no perf penalty due to this change right? Would you consider to prefetch the addr before the db logic just to be on t= he safe side? > volatile uint64_t *src =3D ((volatile uint64_t *)wqe); >=20 > rte_cio_wmb(); > diff --git a/drivers/net/mlx5/mlx5_trigger.c > b/drivers/net/mlx5/mlx5_trigger.c index 7c1e5594d6..b7fde35758 100644 > --- a/drivers/net/mlx5/mlx5_trigger.c > +++ b/drivers/net/mlx5/mlx5_trigger.c > @@ -58,12 +58,6 @@ mlx5_txq_start(struct rte_eth_dev *dev) > goto error; > } > } > - ret =3D mlx5_tx_uar_remap(dev, priv->sh->ctx->cmd_fd); > - if (ret) { > - /* Adjust index for rollback. */ > - i =3D priv->txqs_n - 1; > - goto error; > - } > return 0; > error: > ret =3D rte_errno; /* Save rte_errno before cleanup. */ diff --git > a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index > 4bd08cb035..5fb1761955 100644 > --- a/drivers/net/mlx5/mlx5_txq.c > +++ b/drivers/net/mlx5/mlx5_txq.c > @@ -229,13 +229,99 @@ mlx5_tx_queue_release(void *dpdk_txq) > } > } >=20 > +/** > + * Initialize Tx UAR registers for primary process. > + * > + * @param txq_ctrl > + * Pointer to Tx queue control structure. > + */ > +static void > +txq_uar_init(struct mlx5_txq_ctrl *txq_ctrl) { > + struct mlx5_priv *priv =3D txq_ctrl->priv; > + struct mlx5_proc_priv *ppriv =3D MLX5_PROC_PRIV(PORT_ID(priv)); > + > + assert(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); > + assert(ppriv); > + ppriv->uar_table[txq_ctrl->txq.idx] =3D txq_ctrl->bf_reg; #ifndef > +RTE_ARCH_64 > + struct mlx5_priv *priv =3D txq_ctrl->priv; > + struct mlx5_txq_data *txq =3D &txq_ctrl->txq; > + unsigned int lock_idx; > + /* Assign an UAR lock according to UAR page number */ > + lock_idx =3D (txq_ctrl->uar_mmap_offset / page_size) & > + MLX5_UAR_PAGE_NUM_MASK; > + txq->uar_lock =3D &priv->uar_lock[lock_idx]; #endif } >=20 > /** > - * Mmap TX UAR(HW doorbell) pages into reserved UAR address space. > - * Both primary and secondary process do mmap to make UAR address > - * aligned. > + * Remap UAR register of a Tx queue for secondary process. > * > - * @param[in] dev > + * Remapped address is stored at the table in the process private > +structure of > + * the device, indexed by queue index. > + * > + * @param txq_ctrl > + * Pointer to Tx queue control structure. > + * @param fd > + * Verbs file descriptor to map UAR pages. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +static int > +txq_uar_init_secondary(struct mlx5_txq_ctrl *txq_ctrl, int fd) { > + struct mlx5_priv *priv =3D txq_ctrl->priv; > + struct mlx5_proc_priv *ppriv =3D MLX5_PROC_PRIV(PORT_ID(priv)); > + struct mlx5_txq_data *txq =3D &txq_ctrl->txq; > + void *addr; > + uintptr_t uar_va; > + uintptr_t offset; > + const size_t page_size =3D sysconf(_SC_PAGESIZE); > + > + assert(ppriv); > + /* > + * As rdma-core, UARs are mapped in size of OS page > + * size. Ref to libmlx5 function: mlx5_init_context() > + */ > + uar_va =3D (uintptr_t)txq_ctrl->bf_reg; > + offset =3D uar_va & (page_size - 1); /* Offset in page. */ > + addr =3D mmap(NULL, page_size, PROT_WRITE, MAP_SHARED, fd, > + txq_ctrl->uar_mmap_offset); > + if (addr =3D=3D MAP_FAILED) { > + DRV_LOG(ERR, > + "port %u mmap failed for BF reg of txq %u", > + txq->port_id, txq->idx); > + rte_errno =3D ENXIO; > + return -rte_errno; > + } > + addr =3D RTE_PTR_ADD(addr, offset); > + ppriv->uar_table[txq->idx] =3D addr; > + return 0; > +} > + > +/** > + * Unmap UAR register of a Tx queue for secondary process. > + * > + * @param txq_ctrl > + * Pointer to Tx queue control structure. > + */ > +static void > +txq_uar_uninit_secondary(struct mlx5_txq_ctrl *txq_ctrl) { > + struct mlx5_proc_priv *ppriv =3D MLX5_PROC_PRIV(PORT_ID(txq_ctrl- > >priv)); > + const size_t page_size =3D sysconf(_SC_PAGESIZE); > + void *addr; > + > + addr =3D ppriv->uar_table[txq_ctrl->txq.idx]; > + munmap(RTE_PTR_ALIGN_FLOOR(addr, page_size), page_size); } > + > +/** > + * Initialize Tx UAR registers for secondary process. > + * > + * @param dev > * Pointer to Ethernet device. > * @param fd > * Verbs file descriptor to map UAR pages. > @@ -244,81 +330,36 @@ mlx5_tx_queue_release(void *dpdk_txq) > * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > int > -mlx5_tx_uar_remap(struct rte_eth_dev *dev, int fd) > +mlx5_tx_uar_init_secondary(struct rte_eth_dev *dev, int fd) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > - unsigned int i, j; > - uintptr_t pages[priv->txqs_n]; > - unsigned int pages_n =3D 0; > - uintptr_t uar_va; > - uintptr_t off; > - void *addr; > - void *ret; > struct mlx5_txq_data *txq; > struct mlx5_txq_ctrl *txq_ctrl; > - int already_mapped; > - size_t page_size =3D sysconf(_SC_PAGESIZE); > -#ifndef RTE_ARCH_64 > - unsigned int lock_idx; > -#endif > + unsigned int i; > + int ret; >=20 > - memset(pages, 0, priv->txqs_n * sizeof(uintptr_t)); > - /* > - * As rdma-core, UARs are mapped in size of OS page size. > - * Use aligned address to avoid duplicate mmap. > - * Ref to libmlx5 function: mlx5_init_context() > - */ > + assert(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > for (i =3D 0; i !=3D priv->txqs_n; ++i) { > if (!(*priv->txqs)[i]) > continue; > txq =3D (*priv->txqs)[i]; > txq_ctrl =3D container_of(txq, struct mlx5_txq_ctrl, txq); > assert(txq->idx =3D=3D (uint16_t)i); > - /* UAR addr form verbs used to find dup and offset in page. > */ > - uar_va =3D (uintptr_t)txq_ctrl->bf_reg_orig; > - off =3D uar_va & (page_size - 1); /* offset in page. */ > - uar_va =3D RTE_ALIGN_FLOOR(uar_va, page_size); /* page > addr. */ > - already_mapped =3D 0; > - for (j =3D 0; j !=3D pages_n; ++j) { > - if (pages[j] =3D=3D uar_va) { > - already_mapped =3D 1; > - break; > - } > - } > - /* new address in reserved UAR address space. */ > - addr =3D RTE_PTR_ADD(mlx5_shared_data->uar_base, > - uar_va & (uintptr_t)(MLX5_UAR_SIZE - 1)); > - if (!already_mapped) { > - pages[pages_n++] =3D uar_va; > - /* fixed mmap to specified address in reserved > - * address space. > - */ > - ret =3D mmap(addr, page_size, > - PROT_WRITE, MAP_FIXED | MAP_SHARED, > fd, > - txq_ctrl->uar_mmap_offset); > - if (ret !=3D addr) { > - /* fixed mmap have to return same address > */ > - DRV_LOG(ERR, > - "port %u call to mmap failed on UAR" > - " for txq %u", > - dev->data->port_id, txq->idx); > - rte_errno =3D ENXIO; > - return -rte_errno; > - } > - } > - if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) /* save > once */ > - txq_ctrl->txq.bf_reg =3D RTE_PTR_ADD((void *)addr, > off); > - else > - assert(txq_ctrl->txq.bf_reg =3D=3D > - RTE_PTR_ADD((void *)addr, off)); > -#ifndef RTE_ARCH_64 > - /* Assign a UAR lock according to UAR page number */ > - lock_idx =3D (txq_ctrl->uar_mmap_offset / page_size) & > - MLX5_UAR_PAGE_NUM_MASK; > - txq->uar_lock =3D &priv->uar_lock[lock_idx]; > -#endif > + ret =3D txq_uar_init_secondary(txq_ctrl, fd); > + if (ret) > + goto error; > } > return 0; > +error: > + /* Rollback. */ > + do { > + if (!(*priv->txqs)[i]) > + continue; > + txq =3D (*priv->txqs)[i]; > + txq_ctrl =3D container_of(txq, struct mlx5_txq_ctrl, txq); > + txq_uar_uninit_secondary(txq_ctrl); > + } while (i--); > + return -rte_errno; > } >=20 > /** > @@ -507,7 +548,6 @@ mlx5_txq_ibv_new(struct rte_eth_dev *dev, > uint16_t idx) > txq_data->wqes =3D qp.sq.buf; > txq_data->wqe_n =3D log2above(qp.sq.wqe_cnt); > txq_data->qp_db =3D &qp.dbrec[MLX5_SND_DBR]; > - txq_ctrl->bf_reg_orig =3D qp.bf.reg; > txq_data->cq_db =3D cq_info.dbrec; > txq_data->cqes =3D > (volatile struct mlx5_cqe (*)[]) > @@ -521,6 +561,8 @@ mlx5_txq_ibv_new(struct rte_eth_dev *dev, > uint16_t idx) > txq_ibv->qp =3D tmpl.qp; > txq_ibv->cq =3D tmpl.cq; > rte_atomic32_inc(&txq_ibv->refcnt); > + txq_ctrl->bf_reg =3D qp.bf.reg; > + txq_uar_init(txq_ctrl); > if (qp.comp_mask & MLX5DV_QP_MASK_UAR_MMAP_OFFSET) { > txq_ctrl->uar_mmap_offset =3D qp.uar_mmap_offset; > DRV_LOG(DEBUG, "port %u: uar_mmap_offset 0x%lx", @@ - > 778,6 +820,7 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, > uint16_t desc, > tmpl->priv =3D priv; > tmpl->socket =3D socket; > tmpl->txq.elts_n =3D log2above(desc); > + tmpl->txq.port_id =3D dev->data->port_id; > tmpl->txq.idx =3D idx; > txq_set_params(tmpl); > DRV_LOG(DEBUG, "port %u device_attr.max_qp_wr is %d", @@ - > 836,15 +879,12 @@ mlx5_txq_release(struct rte_eth_dev *dev, uint16_t idx) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > struct mlx5_txq_ctrl *txq; > - size_t page_size =3D sysconf(_SC_PAGESIZE); >=20 > if (!(*priv->txqs)[idx]) > return 0; > txq =3D container_of((*priv->txqs)[idx], struct mlx5_txq_ctrl, txq); > if (txq->ibv && !mlx5_txq_ibv_release(txq->ibv)) > txq->ibv =3D NULL; > - munmap((void *)RTE_ALIGN_FLOOR((uintptr_t)txq->txq.bf_reg, > page_size), > - page_size); > if (rte_atomic32_dec_and_test(&txq->refcnt)) { > txq_free_elts(txq); > mlx5_mr_btree_free(&txq->txq.mr_ctrl.cache_bh); > -- > 2.11.0