From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0051.outbound.protection.outlook.com [104.47.2.51]) by dpdk.org (Postfix) with ESMTP id E6B5EAAC6 for ; Tue, 5 Jun 2018 02:35:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xFBbOT3JSKOD0oVS+IfnAe6zAeDkzPoMP55wGHO3qa0=; b=lL84PSaH1HT0c3UMviaHJRgVeYqKfhsvU73yHoH1l9yzLH/rZ8acLIzoXcfAWEp8AMz7GdWkq7cfYQ60BgHVxshsDbrLjjASbbSgeJoDHkzPu4ifPbeaReWfSi+CgLJcohgG0jPSfzOmG5uTQHROSYWaRMGF6Zgv9k9JQBvi+ms= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by VI1PR0501MB2048.eurprd05.prod.outlook.com (2603:10a6:800:36::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.12; Tue, 5 Jun 2018 00:35:45 +0000 From: Yongseok Koh To: yliu@fridaylinux.org Cc: stable@dpdk.org, shahafs@mellanox.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Xueming Li Date: Mon, 4 Jun 2018 17:35:09 -0700 Message-Id: <20180605003518.14227-1-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180605002732.13866-1-yskoh@mellanox.com> References: <20180605002732.13866-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: BYAPR07CA0036.namprd07.prod.outlook.com (2603:10b6:a02:bc::49) To VI1PR0501MB2048.eurprd05.prod.outlook.com (2603:10a6:800:36::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0501MB2048; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 3:4NV2H8xJhwZUc7hwfK94Sl0TbSpsYHtAFzPvuvjTIphbAoDDw3vhUHyft7VtWh2ALJ2AcdqEXbyDi4ezrs9qG8ajAzwLkq3yTrONutsGAj5fZqwILHEGcIdFPRC6H+K/0m5UZZYNRIL75D9gPC1k3IbztG8dpHwmfzg29tJ/d4EQuI83gGAhRmd1rZlMc18XS7x35gpsvyRgmKK7+ao5hU93m1TzFExhgbc72toICVZsESriUmYRnYTP1wPThntc; 25:Qur2QAqE8hZ5+ULzZ/WINAnjY0idA1BE/VVIBNhxs9pIy8pBZnvxQc413JdUcd6CAZm+eFYwu372fjD8qXjsXn3EEPMgmyOuKcJcMKvH4ztfr3hyxD28ZyQ6Cs3msqW12ZwNGHuBZwxz8R+lLEdymNvzz5iKvuE/CX3pl67c/1RkwNA/I9Y95wgxBextnd2YRezWil8qmwsKPoK6P99hdqXhv5Aob5n7URJp9q4A6SfnTyYsg8Rv/3yg2lN87ImZ3rOtD7eaOxZSh6iV2LHXRRhC0wHvUdmAtBs7M77F3CLtVbf3DYQjpYk7R1T8r4sYQKxuTVDlousgMS+tp1jhiQ==; 31:vsuQMEpMqDQZfjq8XerufUxXvs5T5O0CfdudG4+qzHMEgRMHrPqfKpH2aeci6ATxRlF0s6Nnt4WZ6ovMf+uZYmd2DMzSGeAOu4P3wBzsPCaAPP8goF2eZ/eiJSVBJrnHtSREMdiqyC3YKenkdIsbwV+ZiwulgTx9qOow4RpV/uwtyd8BiOePZL1wtkSB5Rg1qsIC9hJu5+5A7PVlPiPRSmyDLp1LN40kFIb2fdic5dE= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2048: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 20:16KT4dn9NxXepxBv19yGA+5vNGnMHkXtnfHdL/D4Po45RmxWUqaBmeyJ/O9aKcKTh858uNdqdyKpvz0BHHHJ9TcTJ+xk9e6dz9lFEWuJVR9us8Al6f0Ljt+Ez2EurGZM7OTcm4EcfapTTrZNvrzME8vswjA3TbUz1mVJiNwrWNaPBQObGKujTM5GOkXVNHuf7ueqMJtXgcVFbmy51aQCDDXtT5D5LKjm/l6AwTrTITbyFx9lYyC9C9w1sZAmZjrHP2xlTxYcJSRJgkfHm5M13d71JxP0v/cuZdNtUsATgpzn+5tfOZ1GSEpEIyhLpPiKwC2MzuXgybz9QQYA7LexY/fBP2NOUg3YDq3kSpWNY3AVo782Af8hhUq99faq2aViv954rF9RyVUeigW6Ryi0kcGSL+Sa7uV6vx2/SAipt6/eFXTPA/bvR+lJvzuMfk05rdnYMEfqQ6pPFBgDKZokb0wi6Nwcwz5+W/sm9CbO1VDLbVpcI1LXOFjxuKWvHkry; 4:My/YN4vcMN2OhCj4D5QwWznRAiJPnNALffaX7Seo3OjrX9nu2X/ACwpft6+O3Ma5GYaEVZ/I1bPbmG9U6ExeOUrVlqHCS1RsOA9Z20OovFZaakpjaLM54CipPij0T+WhDM2XYN6l0+cSgf67wfXfPCDyNntmOOGboUf6pJHorUTeHQEwY+uvNNRxp3b5G953hoJxbgyNXbScs2bdcU21Da928H5/2ndVrmHquEtp+P2vIkzO0JYRPvbKd55I7DnkuhW85X0qR+kzueVDj9bn+A== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:VI1PR0501MB2048; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2048; X-Forefront-PRVS: 0694C54398 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(346002)(396003)(376002)(39860400002)(39380400002)(199004)(189003)(3846002)(6666003)(2351001)(1076002)(6116002)(316002)(386003)(59450400001)(26005)(52116002)(186003)(2361001)(16526019)(68736007)(51416003)(7696005)(76176011)(48376002)(446003)(16586007)(55016002)(305945005)(956004)(11346002)(476003)(2616005)(486006)(50466002)(21086003)(69596002)(47776003)(53936002)(81156014)(81166006)(8676002)(86362001)(106356001)(8936002)(25786009)(50226002)(7736002)(478600001)(4326008)(5660300001)(36756003)(107886003)(6916009)(97736004)(66066001)(2906002)(105586002)(14583001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2048; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0501MB2048; 23:908O+990uBJFLQjsjPBcObUaDXTwMOe0lq3aE24?= =?us-ascii?Q?zOtWwBw5cIa8PIjHwyO4mm/z442hlNY+RRmM01E34tXWEduDdLqiV/vFhlCV?= =?us-ascii?Q?J1dOZ00FC94Z8XN4svZKt5hn4nNnnJKUCui+NvTTYPs/GKqpIE840iU0QSj/?= =?us-ascii?Q?ambhZoHBeCsvPGS+Ju1BuVshXayTHl4Yu4jmCyy7IpxVOC8PR6OHvffVqKu3?= =?us-ascii?Q?4lGYxu5sFAJUuxh7PeRyouDWQxEmq+0dn2EoRj4ikDDDTBZOeBL6LtKyyOJk?= =?us-ascii?Q?Ppft6bnwFowrAmEYoBoIzO3zSPNRdmKSGFS3uKWJ9AlaLl21LN5UjPc9o3wA?= =?us-ascii?Q?B0ieYVHYFonbOb0sCJG8vl3kGLytW2sC1vZcK+ZPkhirK7rbHk9KoZy+v5vE?= =?us-ascii?Q?4/9jIkfNDMcjgC3LyNaXtoGrXXIWb3BrBPJtmfyCMB2k+Fl9mEIjoloNDbFU?= =?us-ascii?Q?gjwbyfw9Kln2vLGYKaoeGfcVt6nLpA/8nSJVSIuOhui2tVFf5P+bnMhKwhFD?= =?us-ascii?Q?RxVIuoudUNX7RGLWkWilnxvIsa74iriZAdgab97RzYNoS9jPaRuKIIHw9FJN?= =?us-ascii?Q?5ubUAlf71lujRogGw08p+sfia61nH/gW6eigObOUFXKXBWeiJWoHxeYN7wrn?= =?us-ascii?Q?9ZucP4rleJidgvNJ1M7c8OqnhojbDpqb8C4ddFl2wGPlFbDY2nC709hwd2r+?= =?us-ascii?Q?E2WmoA2P6Omn6G5zgf9sHucO0A55U6hgrbYLqTMsR72eR1XcciROmwR1JzoO?= =?us-ascii?Q?sxayEftKxOi2XlH+qhN4q8woztpL3keiAuDJxoqMWeOY1GwPIsIFMlwYbidi?= =?us-ascii?Q?+LKpP1oJsvDtyAkLyo/BNICXeALRmQatWt0loajvoKOSCKfFJJQTPKNkjkzl?= =?us-ascii?Q?4bUlr1L63h44wVmyMAMeNyTh49muhOqApKfmI8tX2lipvGPqpAAkPcud+bw1?= =?us-ascii?Q?jWsdMkKL5lYmacH63fH4HhlehdMScNn0ZcmOGdkfFiLwZN54LreAyQLqZ2oc?= =?us-ascii?Q?8uYO/+sLICvVS93/7T/1gExV5N83FhtknxMHBFhsmLRb+Zttw+1Rqbwce9ie?= =?us-ascii?Q?zdrATGZfRb/TwwlTpCQoyDCpdD6j7eWiDT3Vnuir2vNgbj0Ztf+W5fYpu296?= =?us-ascii?Q?OMrtgZN5WKG/t8YmIPbZhS4vVKcInHVQkkhkDpPkPl8xEmHvTojn6SDYPZtt?= =?us-ascii?Q?onDEIdY3LRvrNJA5GC0k0QLjCJbXx4eNs7sJ3XbwlRJFBIVxD5PdcJKrRYBK?= =?us-ascii?Q?s0alLnyIoXkij28jVoIJD3Ic98r/fAk7oamhfLahgi0069Nc6Ww0QJZRFDhf?= =?us-ascii?Q?/HFRCYAchx0dr2QqGqsYbfu1Hd8FAPCdPVGBet0+lUNoR?= X-Microsoft-Antispam-Message-Info: 62VatePYysFvrwfyj6zxwvNKUFs8G/bqJ6P+vklOATEeLZC2wzHBQmsodWEvTuqvLldGB7hcvr8ahuDmleYS47GNkti6Ui75lrxePdBUgUI6JB+2n+1cdxChpRU++T8XGX7NdXcVT6WwDXyeplGLZT9xYdDalMxGUkoa7oWSbrbLAsUBQXHRWHEGA/noX27V X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 6:UixAgcrbpvnXhD+l+mP6O6IvK+NN5m/YKQHHlYPH4WH0NohDGMB2rlNp2hSiCQJdHq0aYgB+d8yFedGN9tPBK+kYl1S/Ca0sRieCa3nuYYT6QVVh2k27vXM7R1blaZfUiK9dWw0R5XRbu6+/twE+E12jHGiprij/CtBd5lInWIIy5zjvJqZfqszwmftmhT4wTe7pDCInaXeLtIfAGTHgPn4AlVGi+4hQavWMuE33x2cffOTVwMefPuldstoj5atifPCsZyyO+ecJmrLtsugrUh31OPU83yM13WAvzo2otofQBGQxab/irhk1MtqwPD/jjHYN7TPJO5qEdT+ncIzcFzG5XvRMG6dUgYW/RUgsn4U51bMSANl/0zSbFnz/TNVuAucbWl10GLyUJYTOhpzD2Yfw/TkStLRGJG3sfWxG7eHBUb6B9eUvc9XQtZj2MKC7PxH7w8voUMVhNhE/6P32bg==; 5:h7Nc/RzRZTbiv/YSAOMFM8mmTcrLxP+POvrsEnh8AXClxN1K0irChd6KJRmb57YIHHMeYwGBObg02SvGZHcPzHfb/EbSTG3/468IIGBHusFfZJ83uvzyaFys+c2tlbH+WTnb/8CTlvGB+N91ZrPBf+07Pt+ztNM6Dw1aUoqUfgY=; 24:ztWLKEq+bn4JDvDaOevi37r3mkfB+F06ze/8Nrag9qoU5DzzM9xLB1KlhjHSKj6bDWaB4q3MUby27F+vtLjmK9NdphaicofzJ345wRlgUOY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 7:gjmJ/1xM7RLryxdZdQ8igZydfslXS4h/xmmMxVNMmeJV4iYH6KXBsyxg81FzQqSN58L5Uue/DpoqPdQ1uSW8JM+W17JU3asH3uESIjm0sFhfG5PSbkD7RyVlmDFs29QHDeALb6W5CqqUHcnHJV/L96W7vj9yV5Tw2lS6/tbVbjxCkn5DWvXPyMxniLWYpulq9z0mzUr9cROya0HEnr70PrgnFZqAA9zRmTcrB6Ncc/kpGQ8c5HABdHrEZqY50YPi X-MS-Office365-Filtering-Correlation-Id: bb062555-1a30-4e03-5a73-08d5ca7c4797 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2018 00:35:45.3961 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb062555-1a30-4e03-5a73-08d5ca7c4797 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2048 Subject: [dpdk-stable] [PATCH v2 10/67] net/mlx5: map UAR address around huge pages X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 00:35:49 -0000 From: Xueming Li [ backported from upstream commit 4a984153430cbdafe2b971e999bba8d96610b299 ] Reserving the memory space for the UAR near huge pages helps to **reduce** the cases where the secondary process cannot start. Those pages being physical pages they must be mapped at the same virtual address as in the primary process to have a working secondary process. As this remap is almost the latest being done by the processes (libraries, heaps, stacks are already loaded), similar to huge pages, there is **no guarantee** this mechanism will always work. Signed-off-by: Xueming Li Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 114 ++++++++++++++++++++++++++++++++++++++-- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 10 ++++ drivers/net/mlx5/mlx5_rxtx.h | 3 +- drivers/net/mlx5/mlx5_trigger.c | 7 ++- drivers/net/mlx5/mlx5_txq.c | 51 +++++++++++++----- 6 files changed, 167 insertions(+), 19 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 16ba5b724..7dc3dc399 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -39,6 +39,7 @@ #include #include #include +#include /* Verbs header. */ /* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ @@ -56,6 +57,7 @@ #include #include #include +#include #include #include "mlx5.h" @@ -496,6 +498,106 @@ mlx5_args(struct mlx5_args *args, struct rte_devargs *devargs) static struct rte_pci_driver mlx5_driver; +/* + * Reserved UAR address space for TXQ UAR(hw doorbell) mapping, process + * local resource used by both primary and secondary to avoid duplicate + * reservation. + * The space has to be available on both primary and secondary process, + * TXQ UAR maps to this area using fixed mmap w/o double check. + */ +static void *uar_base; + +/** + * Reserve UAR address space for primary process. + * + * @param[in] priv + * Pointer to private structure. + * + * @return + * 0 on success, errno value on failure. + */ +static int +priv_uar_init_primary(struct priv *priv) +{ + void *addr = (void *)0; + int i; + const struct rte_mem_config *mcfg; + int ret; + + if (uar_base) { /* UAR address space mapped. */ + priv->uar_base = uar_base; + return 0; + } + /* find out lower bound of hugepage segments */ + mcfg = rte_eal_get_configuration()->mem_config; + for (i = 0; i < RTE_MAX_MEMSEG && mcfg->memseg[i].addr; i++) { + if (addr) + addr = RTE_MIN(addr, mcfg->memseg[i].addr); + else + addr = mcfg->memseg[i].addr; + } + /* keep distance to hugepages to minimize potential conflicts. */ + addr = RTE_PTR_SUB(addr, MLX5_UAR_OFFSET + MLX5_UAR_SIZE); + /* anonymous mmap, no real memory consumption. */ + addr = mmap(addr, MLX5_UAR_SIZE, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ERROR("Failed to reserve UAR address space, please adjust " + "MLX5_UAR_SIZE or try --base-virtaddr"); + ret = ENOMEM; + return ret; + } + /* Accept either same addr or a new addr returned from mmap if target + * range occupied. + */ + INFO("Reserved UAR address space: %p", addr); + priv->uar_base = addr; /* for primary and secondary UAR re-mmap. */ + uar_base = addr; /* process local, don't reserve again. */ + return 0; +} + +/** + * Reserve UAR address space for secondary process, align with + * primary process. + * + * @param[in] priv + * Pointer to private structure. + * + * @return + * 0 on success, errno value on failure. + */ +static int +priv_uar_init_secondary(struct priv *priv) +{ + void *addr; + int ret; + + assert(priv->uar_base); + if (uar_base) { /* already reserved. */ + assert(uar_base == priv->uar_base); + return 0; + } + /* anonymous mmap, no real memory consumption. */ + addr = mmap(priv->uar_base, MLX5_UAR_SIZE, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ERROR("UAR mmap failed: %p size: %llu", + priv->uar_base, MLX5_UAR_SIZE); + ret = ENXIO; + return ret; + } + if (priv->uar_base != addr) { + ERROR("UAR address %p size %llu occupied, please adjust " + "MLX5_UAR_OFFSET or try EAL parameter --base-virtaddr", + priv->uar_base, MLX5_UAR_SIZE); + ret = ENXIO; + return ret; + } + uar_base = addr; /* process local, don't reserve again */ + INFO("Reserved UAR address space: %p", addr); + return 0; +} + /** * Assign parameters from args into priv, only non default * values are considered. @@ -710,6 +812,11 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev->device = &pci_dev->device; eth_dev->dev_ops = &mlx5_dev_sec_ops; priv = eth_dev->data->dev_private; + err = priv_uar_init_secondary(priv); + if (err < 0) { + err = -err; + goto error; + } /* Receive command fd from primary process */ err = priv_socket_connect(priv); if (err < 0) { @@ -718,10 +825,8 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } /* Remap UAR for Tx queues. */ err = priv_tx_uar_remap(priv, err); - if (err < 0) { - err = -err; + if (err) goto error; - } priv_dev_select_rx_function(priv, eth_dev); priv_dev_select_tx_function(priv, eth_dev); continue; @@ -883,6 +988,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) WARN("Rx CQE compression isn't supported"); priv->cqe_comp = 0; } + err = priv_uar_init_primary(priv); + if (err) + goto port_error; /* Configure the first MAC address by default. */ if (priv_get_mac(priv, &mac.addr_bytes)) { ERROR("cannot get MAC address, is mlx5_en loaded?" diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 004e83670..8f49cecc8 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -166,6 +166,7 @@ struct priv { struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ rte_spinlock_t lock; /* Lock for control functions. */ int primary_socket; /* Unix socket for primary process. */ + void *uar_base; /* Reserved address space for UAR mapping */ struct rte_intr_handle intr_handle_socket; /* Interrupt handler. */ struct mlx5_verbs_alloc_ctx verbs_alloc_ctx; /* Context for Verbs allocator. */ diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 24caf7e79..9c740f5e1 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -113,4 +113,14 @@ /* Maximum number of attempts to query link status before giving up. */ #define MLX5_MAX_LINK_QUERY_ATTEMPTS 5 +/* Reserved address space for UAR mapping. */ +#define MLX5_UAR_SIZE (1ULL << 32) + +/* Offset of reserved UAR address space to hugepage memory. Offset is used here + * to minimize possibility of address next to hugepage being used by other code + * in either primary or secondary process, failing to map TX UAR would make TX + * packets invisible to HW. + */ +#define MLX5_UAR_OFFSET (1ULL << 32) + #endif /* RTE_PMD_MLX5_DEFS_H_ */ diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index e35a6ab34..28761c299 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -204,7 +204,7 @@ struct mlx5_txq_data { volatile void *wqes; /* Work queue (use volatile to write into). */ volatile uint32_t *qp_db; /* Work queue doorbell. */ volatile uint32_t *cq_db; /* Completion queue doorbell. */ - volatile void *bf_reg; /* Blueflame register. */ + volatile void *bf_reg; /* Blueflame register remapped. */ struct mlx5_mr *mp2mr[MLX5_PMD_TX_MP_CACHE]; /* MR translation table. */ struct rte_mbuf *(*elts)[]; /* TX elements. */ struct mlx5_txq_stats stats; /* TX queue counters. */ @@ -229,6 +229,7 @@ struct mlx5_txq_ctrl { struct mlx5_txq_ibv *ibv; /* Verbs queue object. */ struct mlx5_txq_data txq; /* Data path structure. */ off_t uar_mmap_offset; /* UAR mmap offset for non-primary process. */ + volatile void *bf_reg_orig; /* Blueflame register from verbs. */ }; /* mlx5_rxq.c */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index d682ea2c9..da9494eb8 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -76,10 +76,13 @@ priv_txq_start(struct priv *priv) goto error; } } - return -ret; + ret = priv_tx_uar_remap(priv, priv->ctx->cmd_fd); + if (ret) + goto error; + return ret; error: priv_txq_stop(priv); - return -ret; + return ret; } static void diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 12f258bbb..f39b98a9a 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -217,7 +217,9 @@ mlx5_tx_queue_release(void *dpdk_txq) /** - * Map locally UAR used in Tx queues for BlueFlame doorbell. + * Mmap TX UAR(HW doorbell) pages into reserved UAR address space. + * Both primary and secondary process do mmap to make UAR address + * aligned. * * @param[in] priv * Pointer to private structure. @@ -234,11 +236,14 @@ priv_tx_uar_remap(struct priv *priv, int fd) uintptr_t pages[priv->txqs_n]; unsigned int pages_n = 0; uintptr_t uar_va; + uintptr_t off; void *addr; + void *ret; struct mlx5_txq_data *txq; struct mlx5_txq_ctrl *txq_ctrl; int already_mapped; size_t page_size = sysconf(_SC_PAGESIZE); + int r; memset(pages, 0, priv->txqs_n * sizeof(uintptr_t)); /* @@ -251,8 +256,10 @@ priv_tx_uar_remap(struct priv *priv, int fd) continue; txq = (*priv->txqs)[i]; txq_ctrl = container_of(txq, struct mlx5_txq_ctrl, txq); - uar_va = (uintptr_t)txq_ctrl->txq.bf_reg; - uar_va = RTE_ALIGN_FLOOR(uar_va, page_size); + /* UAR addr form verbs used to find dup and offset in page. */ + uar_va = (uintptr_t)txq_ctrl->bf_reg_orig; + off = uar_va & (page_size - 1); /* offset in page. */ + uar_va = RTE_ALIGN_FLOOR(uar_va, page_size); /* page addr. */ already_mapped = 0; for (j = 0; j != pages_n; ++j) { if (pages[j] == uar_va) { @@ -260,16 +267,30 @@ priv_tx_uar_remap(struct priv *priv, int fd) break; } } - if (already_mapped) - continue; - pages[pages_n++] = uar_va; - addr = mmap((void *)uar_va, page_size, - PROT_WRITE, MAP_FIXED | MAP_SHARED, fd, - txq_ctrl->uar_mmap_offset); - if (addr != (void *)uar_va) { - ERROR("call to mmap failed on UAR for txq %d\n", i); - return -1; + /* new address in reserved UAR address space. */ + addr = RTE_PTR_ADD(priv->uar_base, + uar_va & (MLX5_UAR_SIZE - 1)); + if (!already_mapped) { + pages[pages_n++] = uar_va; + /* fixed mmap to specified address in reserved + * address space. + */ + ret = mmap(addr, page_size, + PROT_WRITE, MAP_FIXED | MAP_SHARED, fd, + txq_ctrl->uar_mmap_offset); + if (ret != addr) { + /* fixed mmap have to return same address */ + ERROR("call to mmap failed on UAR for txq %d\n", + i); + r = ENXIO; + return r; + } } + if (rte_eal_process_type() == RTE_PROC_PRIMARY) /* save once */ + txq_ctrl->txq.bf_reg = RTE_PTR_ADD((void *)addr, off); + else + assert(txq_ctrl->txq.bf_reg == + RTE_PTR_ADD((void *)addr, off)); } return 0; } @@ -416,7 +437,7 @@ mlx5_priv_txq_ibv_new(struct priv *priv, uint16_t idx) txq_data->wqes = qp.sq.buf; txq_data->wqe_n = log2above(qp.sq.wqe_cnt); txq_data->qp_db = &qp.dbrec[MLX5_SND_DBR]; - txq_data->bf_reg = qp.bf.reg; + txq_ctrl->bf_reg_orig = qp.bf.reg; txq_data->cq_db = cq_info.dbrec; txq_data->cqes = (volatile struct mlx5_cqe (*)[]) @@ -717,6 +738,7 @@ mlx5_priv_txq_release(struct priv *priv, uint16_t idx) { unsigned int i; struct mlx5_txq_ctrl *txq; + size_t page_size = sysconf(_SC_PAGESIZE); if (!(*priv->txqs)[idx]) return 0; @@ -736,6 +758,9 @@ mlx5_priv_txq_release(struct priv *priv, uint16_t idx) txq->txq.mp2mr[i] = NULL; } } + if (priv->uar_base) + munmap((void *)RTE_ALIGN_FLOOR((uintptr_t)txq->txq.bf_reg, + page_size), page_size); if (rte_atomic32_dec_and_test(&txq->refcnt)) { txq_free_elts(txq); LIST_REMOVE(txq, next); -- 2.11.0