From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 33CDFA0096 for ; Wed, 10 Apr 2019 21:12:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 30DE51B14D; Wed, 10 Apr 2019 21:12:52 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40050.outbound.protection.outlook.com [40.107.4.50]) by dpdk.org (Postfix) with ESMTP id 4B6E21B148 for ; Wed, 10 Apr 2019 21:12:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6GwTK7qRFKQ++lwkR9w7hsT3wX50PTiwpivvjalOdig=; b=obBXLynskJWm7FTPxTdTzmE8kwV/7fRkb3nCjYla3dVDTnSQxA2582S3/SLyGszXobd9gUzwhvocCH6SeVlIrragsZv712/nNUUysUlw3Z+y0vvFAYJvyCKbzE/xIKmihdvE+G4wFshMuWllqrEvaIc+5eIuqeNzW4yVge2moh4= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB4075.eurprd05.prod.outlook.com (52.134.67.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1792.14; Wed, 10 Apr 2019 19:12:50 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::6072:43be:7c2d:103a]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::6072:43be:7c2d:103a%3]) with mapi id 15.20.1792.009; Wed, 10 Apr 2019 19:12:50 +0000 From: Yongseok Koh To: Ferruh Yigit CC: Shahaf Shuler , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v4 3/4] net/mlx5: remove device register remap Thread-Index: AQHU7ynm6Z95cXAZv0eTutjf1+rlsqY1rVgAgAAXGQA= Date: Wed, 10 Apr 2019 19:12:50 +0000 Message-ID: <235F78D4-58FC-4A3A-AD43-0DC051EB89FF@mellanox.com> References: <20190325193627.19726-1-yskoh@mellanox.com> <20190409231324.35715-1-yskoh@mellanox.com> <20190409231324.35715-4-yskoh@mellanox.com> <69b151d0-f21d-8b5c-89d3-3ceada5a7126@intel.com> In-Reply-To: <69b151d0-f21d-8b5c-89d3-3ceada5a7126@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9ecdb47e-08a6-409b-4231-08d6bde88650 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600139)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:DB3PR0502MB4075; x-ms-traffictypediagnostic: DB3PR0502MB4075: x-microsoft-antispam-prvs: x-forefront-prvs: 00032065B2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(136003)(39860400002)(346002)(396003)(366004)(189003)(199004)(76176011)(478600001)(7736002)(229853002)(186003)(6436002)(36756003)(6486002)(53936002)(6916009)(305945005)(86362001)(33656002)(26005)(316002)(25786009)(3846002)(6512007)(6116002)(54906003)(53546011)(14444005)(102836004)(68736007)(106356001)(256004)(105586002)(6506007)(14454004)(93886005)(81156014)(8676002)(6246003)(81166006)(71190400001)(4326008)(66066001)(71200400001)(83716004)(486006)(2616005)(99286004)(476003)(97736004)(2906002)(82746002)(446003)(8936002)(11346002)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4075; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: toAlnwOwok1G8PmrtYcK8jOD9InEbwf9nBWnRZAsyDftjWuxKZ/RRxJEbAMBkEBf8fhIMlb+eB7OR4qbJKbIp2UGSNFOfxRS2sWXBDgWjlcefo+uYhXemjeo+dEj0hxqjQ7BSEY6qirPrQrYxpbFQzpF/ljyfAcMS6FobDuxHokhFDnHPUQjFvroM0ForRdtl/oFmC7GpZn1y1L50ByrHE3kkNsJiX5XbV+FKNVU8azRZokIibJzeHi1qDCEhpEiQL+fp/hqq3t/B/3oHoFtxu4wBccFSOImi5RWhyIpSTsh3hDjS+n7r6Mt5TdL0/PQNx2lnIgL4Av+QWePXFn+/7Naex3Mc3kRrAu83m2PX1psJR6tzDyH5cBH93JPP9OaFF/6N33Z52VG5ekJg7zQ0KmV5A1TT6NnghMa81CpY5c= Content-Type: text/plain; charset="UTF-8" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ecdb47e-08a6-409b-4231-08d6bde88650 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2019 19:12:50.0861 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4075 Subject: Re: [dpdk-dev] [PATCH v4 3/4] net/mlx5: remove device register remap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190410191250.l9jCOL70VWTuSnQltbngOutj1860G9eX6QzwLbR0qkY@z> > On Apr 10, 2019, at 10:50 AM, Ferruh Yigit wrote= : >=20 > On 4/10/2019 12:13 AM, Yongseok Koh wrote: >> UAR (User Access Region) register does not need to be remapped for prima= ry >> process but it should be remapped only for secondary process. UAR regist= er >> table is in the process private structure in rte_eth_devices[], >> (struct mlx5_proc_priv *)rte_eth_devices[port_id].process_private >>=20 >> The actual UAR table follows the data structure and the table is used fo= r >> both Tx and Rx. >>=20 >> For Tx, BlueFlame in UAR is used to ring the doorbell. MLX5_TX_BFREG(txq= ) >> is defined to get a register for the txq. Processes access its own priva= te >> data to acquire the register from the UAR table. >>=20 >> For Rx, the doorbell in UAR is required in arming CQ event. However, it = is >> a known issue that the register isn't remapped for secondary process. >>=20 >> Signed-off-by: Yongseok Koh >=20 > <...> >=20 >> @@ -229,13 +229,99 @@ mlx5_tx_queue_release(void *dpdk_txq) >> } >> } >>=20 >> +/** >> + * Initialize Tx UAR registers for primary process. >> + * >> + * @param txq_ctrl >> + * Pointer to Tx queue control structure. >> + */ >> +static void >> +txq_uar_init(struct mlx5_txq_ctrl *txq_ctrl) >> +{ >> + struct mlx5_priv *priv =3D txq_ctrl->priv; >> + struct mlx5_proc_priv *ppriv =3D MLX5_PROC_PRIV(PORT_ID(priv)); >> + >> + assert(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); >> + assert(ppriv); >> + ppriv->uar_table[txq_ctrl->txq.idx] =3D txq_ctrl->bf_reg; >> +#ifndef RTE_ARCH_64 >> + struct mlx5_priv *priv =3D txq_ctrl->priv; >> + struct mlx5_txq_data *txq =3D &txq_ctrl->txq; >> + unsigned int lock_idx; >> + /* Assign an UAR lock according to UAR page number */ >> + lock_idx =3D (txq_ctrl->uar_mmap_offset / page_size) & >> + MLX5_UAR_PAGE_NUM_MASK; >> + txq->uar_lock =3D &priv->uar_lock[lock_idx]; >> +#endif >> +} >=20 > This won't compile for arch is not 64bits, since 'page_size' in that bloc= k is > not defined. It is embarrassing that I have committed so many mistakes on this last patc= hset. So many contexts in my head... Or, this patches must be haunted. :-( I always test 32-bit but it looks like a mistake when rebasing it, not sure= ... My apologies. I've sent out v5. For your convenience, here's the diff. $ git diff yskoh/upstr-remove-uar-remap diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 904c4f5c03..6224b3be1a 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -56,16 +56,6 @@ /** Enable extending memsegs when creating a MR. */ #define MLX4_MR_EXT_MEMSEG_EN_KVARG "mr_ext_memseg_en" -/* Reserved address space for UAR mapping. */ -#define MLX4_UAR_SIZE (1ULL << (sizeof(uintptr_t) * 4)) - -/* Offset of reserved UAR address space to hugepage memory. Offset is used= here - * to minimize possibility of address next to hugepage being used by other= code - * in either primary or secondary process, failing to map TX UAR would mak= e TX - * packets invisible to HW. - */ -#define MLX4_UAR_OFFSET (2ULL << (sizeof(uintptr_t) * 4)) - enum { PCI_VENDOR_ID_MELLANOX =3D 0x15b3, }; diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index bfe6655800..69b6960e94 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -91,16 +91,6 @@ /* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10 -/* Reserved address space for UAR mapping. */ -#define MLX5_UAR_SIZE (1ULL << (sizeof(uintptr_t) * 4)) - -/* Offset of reserved UAR address space to hugepage memory. Offset is used= here - * to minimize possibility of address next to hugepage being used by other= code - * in either primary or secondary process, failing to map TX UAR would mak= e TX - * packets invisible to HW. - */ -#define MLX5_UAR_OFFSET (1ULL << (sizeof(uintptr_t) * 4)) - /* Maximum number of UAR pages used by a port, * These are the size and mask for an array of mutexes used to synchronize * the access to port's UARs on platforms that do not support 64 bit write= s. diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 5fb1761955..9965b2b771 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -240,18 +240,19 @@ txq_uar_init(struct mlx5_txq_ctrl *txq_ctrl) { struct mlx5_priv *priv =3D txq_ctrl->priv; struct mlx5_proc_priv *ppriv =3D MLX5_PROC_PRIV(PORT_ID(priv)); +#ifndef RTE_ARCH_64 + unsigned int lock_idx; + const size_t page_size =3D sysconf(_SC_PAGESIZE); +#endif assert(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); assert(ppriv); ppriv->uar_table[txq_ctrl->txq.idx] =3D txq_ctrl->bf_reg; #ifndef RTE_ARCH_64 - struct mlx5_priv *priv =3D txq_ctrl->priv; - struct mlx5_txq_data *txq =3D &txq_ctrl->txq; - unsigned int lock_idx; /* Assign an UAR lock according to UAR page number */ lock_idx =3D (txq_ctrl->uar_mmap_offset / page_size) & MLX5_UAR_PAGE_NUM_MASK; - txq->uar_lock =3D &priv->uar_lock[lock_idx]; + txq_ctrl->txq.uar_lock =3D &priv->uar_lock[lock_idx]; #endif }