From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0049.outbound.protection.outlook.com [104.47.1.49]) by dpdk.org (Postfix) with ESMTP id D606B2C02 for ; Tue, 5 Jun 2018 02:12:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xFBbOT3JSKOD0oVS+IfnAe6zAeDkzPoMP55wGHO3qa0=; b=AarjdyzFqZ41F5ojD84/c2QbucssGGY0xc69ItO2sQd2HDheFqIjrEQ0HEDJDJeiSV0qc3/mAxE8HGU7HUaeAPAXXQefkXeueXAlo+35IbEoa6Hq267Fve2ywJOW3/X1o0IIHiubLK/X6XX/q8HOKbfifQV+AlzZEiOD2L5W1fU= Received: from mellanox.com (209.116.155.178) by AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.11; Tue, 5 Jun 2018 00:12:09 +0000 From: Yongseok Koh To: yliu@fridaylinux.org Cc: stable@dpdk.org, shahafs@mellanox.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Xueming Li Date: Mon, 4 Jun 2018 17:10:32 -0700 Message-Id: <20180605001129.13184-11-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180605001129.13184-1-yskoh@mellanox.com> References: <20180605001129.13184-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: CO1PR15CA0060.namprd15.prod.outlook.com (2603:10b6:101:1f::28) To AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:AM5PR0501MB2036; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 3:YFVJjEv85JMdwgNCDjNObBq3whesmj32HmlVot12IYVFTOhXNUs7up8bHguiJXNuZ6LeRmobbZvChZvI1hpY5uxzNc/rAJvnRDAHLEguctgK9ZVDYA7DmYd6CcG3QsOvTtZoSTPjjAyRSbB3FDJTnhIiN70Pv65YxjVC5sr9LYGk6slr+m+PVX4xcc9kibjq3G0eLIPlaPxvA3X4zhLMSS/qZbY5jbnKYQKKW2siTtX19gvcjuSazGuVccf9sxCT; 25:D/nkHcUufmdljKp9Z8ehpk6O1GPuSzMzsIv4qopF5UjMlsLjFczhVfgQUSL9Y2kSDm3vVAIgXRWYPewzcQrNj8j10Qs5NyWzBoT+E/b5KygxI9NYS40FhhDnKiSFhc8tHylXNDeksmpd57D0u1EXSMflOjXn11coqinORaz/FHe8sfw8DiUMs4o1/VUjdghYqM9WTn7obiRFEL0QWgU++P8wkCkp5NY3PSnX1EVhhW+ezcafNNp6e0RceyLAy3ZSvla8cZ29eRJLBmozQIJP2UNT7RS1ppH4y1umeiZOdjtDNIQwHNUP42zRA5cjzEIZg4KHich0Ga7lRGWqlMRSYw==; 31:LQaQitACuq5gMRtan368LHkfFxGOoxmBhhDGWkAX/iQA6FBeIVUF3gAW5iRi1bVUivlChgg79YUQya12+8YmJNz0/oVwUm8ovDGerSfS6APaYsyc3jVkTik44WzAdZIq5/fDL+swCWgCD/PC/EQ9XWPr97M6i28QwBlxson2BPhPXckfp3OApxrKBRnhaQn4rsXMEWvyu6yPHlX1Zq5ifYMu+bo4uCuvO8j1K/p2j0c= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2036: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 20:p5/4AtbTRt/7LUJivzUUUey20gej+7DdlFiDtQluGTGF6pTtzOgYBAeR1O3bYD2WjiEUyfqBJ9Pps7scuuCDRx3kgfGCWut8P3mfqMVjdMmKcUDEs8vn7hXR6kQIz1nxR/gaMNrHlEX82+Jkik7xDXyVEuIM7nb8bcIaj+I8sMg/Pz36l90ZKZkmRKvvG4xZDF7Ue6uPwxjL/cASYfZhNKAM3n4exSi3ebVOW7Vu7GwYqLIyMaDzaqoXUsOMMpE5yDfuH7fTxXHOXZC6iWu/ALyxZEkKlzPVqkl72Ym/6cXcEdYKoxS4M81EILfLkSteoAueROsqGSNDY/MmHFVyHYW+unlWNaA/Pr0K3s6031M8W8Of/9nUPx7UfCDbawANrCR23EJFoJ5+shACBgdvWKQu46DPvN9U1kOBAFVFW+1OuK3dqMt9qtHe4l3HDtPqYQWbqrEA84trnaiSPUPH2IAo0qzll6OE1QXiUx4ziZFAP5tijG5jf6CUiJ+R+Lgf; 4:4zAwiH7quIRZyPpBR4HKfe1mYzXt11Pie4e12rjp+9Sr1CW0p3WTthxkBjFLB63vvAT6qdFiadWx8rjqdU+40lIqJbYurRJYrml9qGNssFdi3tPuToB0goaxHiKdso1lvmhrDv1uMjG/TRBViZSR8OABLUOLZFXrUOB/vkso+TO/NOiWkXhEyhyQS/3IXT8ACLRXM9Si6lzI5ieylJ0n4Ulv5WNUTjMwB2TVDyjfjh8/nDx9xsI0r/2lBSEKJZkodFt4dvOkGhMEOUKreKq1Vg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231254)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:AM5PR0501MB2036; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0501MB2036; X-Forefront-PRVS: 0694C54398 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39380400002)(366004)(396003)(39860400002)(346002)(376002)(189003)(199004)(478600001)(81156014)(76176011)(51416003)(7696005)(956004)(2361001)(50226002)(305945005)(2616005)(50466002)(47776003)(486006)(97736004)(5660300001)(6666003)(2351001)(106356001)(8936002)(186003)(81166006)(52116002)(6116002)(476003)(6916009)(386003)(446003)(11346002)(59450400001)(66066001)(26005)(3846002)(8676002)(16526019)(1076002)(25786009)(21086003)(86362001)(53936002)(107886003)(4326008)(2906002)(316002)(55016002)(36756003)(48376002)(105586002)(69596002)(68736007)(7736002)(16586007)(14583001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2036; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2036; 23:qDi3GuOdx1bx1WHRLM1hdvIO/o/PYr4Ik6NGbHv?= =?us-ascii?Q?INs5btWfFwrIPNyab9+Byr3c7OY/bnFrBweO3I8Y8ars43zET1dNCaY/kNjp?= =?us-ascii?Q?He6XiWR50EghatZ22dmvqC6m6cSK0HXgwmzzObF/azmyjiNDervWDNqgnPxj?= =?us-ascii?Q?giPIjVzHuxr5NUn1qIQ89nT9PFBkDShNnNg6+f/v5tywyv4ApnQlCeMTZ7WD?= =?us-ascii?Q?wkf19OlWVb7PuEyGrkjxwb/fG5aG8lwNo20hMw0bDrTanzRpDC28UaFz4yok?= =?us-ascii?Q?P/dvMUF6j4q9/fxF5vfQm2QcRXIvLE67zo2JVYhBlIDvZFpjZksYc6SmaaoN?= =?us-ascii?Q?XA/CP+D+DZyDnHIx9RfaTyH/8/n3bnUg/dDaTOQZwv22U/xpVbFrmkui2mtY?= =?us-ascii?Q?4oAD4A4TGltqRde1R6cucqDxUqmuV3oLPFgJIM+miWiBnKxcD2YvtxfGlhjk?= =?us-ascii?Q?KGZQAPgoMXLjFIA5RccNf4Cqf7XY+mq8a6V9DIgtA9d9V0+8ci6stbs1vC+C?= =?us-ascii?Q?BVjzO+kVyzDypcmmaZjzwB8trP5SRoF3ZKE1BWWkUno8GUzeGKmeN2eqdb7E?= =?us-ascii?Q?ora/IjmZ9TBAlkExGV9uLpanFXVf337L0aWfhfvhOyZ5pC1G4DBkDlGHG5wO?= =?us-ascii?Q?8d24iIOM779GZ8SoppftSDktHQH904JtdgO+fe9XLWxJYnqrU6dpPIfA79+W?= =?us-ascii?Q?+CVq5fTRNaH7yZNLqhCikczLft7LOGNmmhMiy8mKtmY2jD95oUBucI5bsUZ9?= =?us-ascii?Q?FiVFTyexYhjmcey6st/JptTuG240Y/KhDNxHq4W91N04wcfVRfLSh4XSrXJC?= =?us-ascii?Q?Kql0cQDTTDORar2cG4jcUwKrfvjobaWRN6PWkJP3mNJJwapkyDGlpwqWl1Ke?= =?us-ascii?Q?CaUOm2Eyl9aNunZ5Xm2O1dhVc6sVvrm3OqssnMY7dJiME1mYwn8K8xv3nao9?= =?us-ascii?Q?QRd/oh4YPoJsOBegmt/a3nZxp5c1TSrq2xDlkSDz+boXye2Dwx+x9YQHvm0n?= =?us-ascii?Q?NZ0aZTpnCsOjtwDRN4Srr/9ycEnJ044s4wUYMZ22viD0clYYfSSnOE9e6UYy?= =?us-ascii?Q?RBMmvNyAnsRs5Ntn2g4rRmLYYcxeUF6z3RVKb/v133NovdtcUVThPvD44hmd?= =?us-ascii?Q?rGK/BO2rge1JABK9R//SSj0mL/ug4Bn5Z5s45a7siDR/n6VTN2+qR9QqP2sa?= =?us-ascii?Q?ao89Dic3Sz1bcyO8d9Q2AgTUClg78D3FmcyiuoP0qthWAhnJpQbQyhthMbZM?= =?us-ascii?Q?rjrBuyxM4oz94Um6DB47w8GilJBWKjelt9DTj79iasRUuiLQWzMlMUPNQcun?= =?us-ascii?Q?N0h2zBozGYhvtVJezKJRFPtgWLZ8J4JV5C+Tb61abxN14?= X-Microsoft-Antispam-Message-Info: vDXcnTLs1AGUzmGs9MvSq8tzzyzTg8zZ7xqJXcXSHBakYMcN6izzN1TlP/5ZWJ5k2EVD8fysq6DeQfbZFJhH+kmHRdAXI6bhHRyZTSDR5YRvzDWuYNhzYHWkbPVZ5r56xP9dWnZIPKcUpWbOdreNT8bnICJYhsZTZQjk+0Ex7ZSsI7uLN1uFxMm9cIWxoiGq X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 6:EOm1hGTkyhmJQpP17Hj0+49yJR5fitC5iZD7JSxaaBmBhZrTXAWLG4nwJNl2T1wUgN8T5DhLdy8JxQ6HJrI0yUcqkr6bT4hSUJS6O+pPE6tzG14qGAk71Bc8tzQa/fo8Tqv2h3JuGYke7koAoaCEIO9+2nXPELiwebdnZLdrsjM1pI2qMDO/l/MEnAVYO1oLfx9C+cA4hd1WDiDbQfMhnrmek3HqJ5PjJlVoQDE/a0KRP0UnlJToCjVju0LaAKVbLdxdt+tjrh1QBfllot+72XKEfO5jN8AWo3vD/F2o3oYpZ5Y5tsW8H1tNXFA+BQ/PvFEzO0fzzRt4HH48Np8/KLFAoz+dS5tyMX5XYZ3yqIv+ibIwNRZccjn43fvEw4jYTC5LnNqBxvhJYSUzBThKxOx5DyMlvUX0ch5cP0rQ6pgnTegfyfVclFY2NrPoruB003Qo/BLZjtFXerpk83tqvQ==; 5:FJbIowphNv+hmgldOP+lwDmh+1viCoKZwC80nDg/p0/nV+mwPX8d6F0c3djaMHTua/iK0eO/apMko8+PltGyBQLItkEabQq3GMA0TqUzXe/txxLEl9QJZJKgSvqWsW0mfWqSDmH4hfgRKEnVE6y+Ru692a0IQJMDF07MkPo3ZPk=; 24:jPWJ94VuV69tZK5XtEhRzK7PyLf8gzcd1wJ6lYIXsopWBqGL2Pwn6WoOvv2BrPYaEPFHd/e6aOrKv2cNTa1lZZtt/C7BBWEzT2uXj4e/7+4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 7:EE5u5Df8qYhewFPVR9exZFk7G2H6dF00HAWCP+T71eb8yYdXf1x/Hl4oP+6RCcfotk/c/AmX2amNP+XkB1p0Gc18fIXGDzcCVaVby9PN/4RojfVn0FK/v0+NohOmBz7uTZE4zx+QWK/Tag8BucFarZWMIGof8rIxcmU+QDUaaT5bh9OMzhLNbiP4spA0czI9IqfEcKndu2uPGQGOcf3ZnRuoo+nriCYJsBEmuZireOjSwsumtSWF+tVQRXOSU3/x X-MS-Office365-Filtering-Correlation-Id: dc124d89-7a93-4f31-049c-08d5ca78fbc7 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2018 00:12:09.6333 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc124d89-7a93-4f31-049c-08d5ca78fbc7 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2036 Subject: [dpdk-stable] [PATCH 10/67] net/mlx5: map UAR address around huge pages X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 00:12:13 -0000 From: Xueming Li [ backported from upstream commit 4a984153430cbdafe2b971e999bba8d96610b299 ] Reserving the memory space for the UAR near huge pages helps to **reduce** the cases where the secondary process cannot start. Those pages being physical pages they must be mapped at the same virtual address as in the primary process to have a working secondary process. As this remap is almost the latest being done by the processes (libraries, heaps, stacks are already loaded), similar to huge pages, there is **no guarantee** this mechanism will always work. Signed-off-by: Xueming Li Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 114 ++++++++++++++++++++++++++++++++++++++-- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 10 ++++ drivers/net/mlx5/mlx5_rxtx.h | 3 +- drivers/net/mlx5/mlx5_trigger.c | 7 ++- drivers/net/mlx5/mlx5_txq.c | 51 +++++++++++++----- 6 files changed, 167 insertions(+), 19 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 16ba5b724..7dc3dc399 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -39,6 +39,7 @@ #include #include #include +#include /* Verbs header. */ /* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */ @@ -56,6 +57,7 @@ #include #include #include +#include #include #include "mlx5.h" @@ -496,6 +498,106 @@ mlx5_args(struct mlx5_args *args, struct rte_devargs *devargs) static struct rte_pci_driver mlx5_driver; +/* + * Reserved UAR address space for TXQ UAR(hw doorbell) mapping, process + * local resource used by both primary and secondary to avoid duplicate + * reservation. + * The space has to be available on both primary and secondary process, + * TXQ UAR maps to this area using fixed mmap w/o double check. + */ +static void *uar_base; + +/** + * Reserve UAR address space for primary process. + * + * @param[in] priv + * Pointer to private structure. + * + * @return + * 0 on success, errno value on failure. + */ +static int +priv_uar_init_primary(struct priv *priv) +{ + void *addr = (void *)0; + int i; + const struct rte_mem_config *mcfg; + int ret; + + if (uar_base) { /* UAR address space mapped. */ + priv->uar_base = uar_base; + return 0; + } + /* find out lower bound of hugepage segments */ + mcfg = rte_eal_get_configuration()->mem_config; + for (i = 0; i < RTE_MAX_MEMSEG && mcfg->memseg[i].addr; i++) { + if (addr) + addr = RTE_MIN(addr, mcfg->memseg[i].addr); + else + addr = mcfg->memseg[i].addr; + } + /* keep distance to hugepages to minimize potential conflicts. */ + addr = RTE_PTR_SUB(addr, MLX5_UAR_OFFSET + MLX5_UAR_SIZE); + /* anonymous mmap, no real memory consumption. */ + addr = mmap(addr, MLX5_UAR_SIZE, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ERROR("Failed to reserve UAR address space, please adjust " + "MLX5_UAR_SIZE or try --base-virtaddr"); + ret = ENOMEM; + return ret; + } + /* Accept either same addr or a new addr returned from mmap if target + * range occupied. + */ + INFO("Reserved UAR address space: %p", addr); + priv->uar_base = addr; /* for primary and secondary UAR re-mmap. */ + uar_base = addr; /* process local, don't reserve again. */ + return 0; +} + +/** + * Reserve UAR address space for secondary process, align with + * primary process. + * + * @param[in] priv + * Pointer to private structure. + * + * @return + * 0 on success, errno value on failure. + */ +static int +priv_uar_init_secondary(struct priv *priv) +{ + void *addr; + int ret; + + assert(priv->uar_base); + if (uar_base) { /* already reserved. */ + assert(uar_base == priv->uar_base); + return 0; + } + /* anonymous mmap, no real memory consumption. */ + addr = mmap(priv->uar_base, MLX5_UAR_SIZE, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ERROR("UAR mmap failed: %p size: %llu", + priv->uar_base, MLX5_UAR_SIZE); + ret = ENXIO; + return ret; + } + if (priv->uar_base != addr) { + ERROR("UAR address %p size %llu occupied, please adjust " + "MLX5_UAR_OFFSET or try EAL parameter --base-virtaddr", + priv->uar_base, MLX5_UAR_SIZE); + ret = ENXIO; + return ret; + } + uar_base = addr; /* process local, don't reserve again */ + INFO("Reserved UAR address space: %p", addr); + return 0; +} + /** * Assign parameters from args into priv, only non default * values are considered. @@ -710,6 +812,11 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev->device = &pci_dev->device; eth_dev->dev_ops = &mlx5_dev_sec_ops; priv = eth_dev->data->dev_private; + err = priv_uar_init_secondary(priv); + if (err < 0) { + err = -err; + goto error; + } /* Receive command fd from primary process */ err = priv_socket_connect(priv); if (err < 0) { @@ -718,10 +825,8 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } /* Remap UAR for Tx queues. */ err = priv_tx_uar_remap(priv, err); - if (err < 0) { - err = -err; + if (err) goto error; - } priv_dev_select_rx_function(priv, eth_dev); priv_dev_select_tx_function(priv, eth_dev); continue; @@ -883,6 +988,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) WARN("Rx CQE compression isn't supported"); priv->cqe_comp = 0; } + err = priv_uar_init_primary(priv); + if (err) + goto port_error; /* Configure the first MAC address by default. */ if (priv_get_mac(priv, &mac.addr_bytes)) { ERROR("cannot get MAC address, is mlx5_en loaded?" diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 004e83670..8f49cecc8 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -166,6 +166,7 @@ struct priv { struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ rte_spinlock_t lock; /* Lock for control functions. */ int primary_socket; /* Unix socket for primary process. */ + void *uar_base; /* Reserved address space for UAR mapping */ struct rte_intr_handle intr_handle_socket; /* Interrupt handler. */ struct mlx5_verbs_alloc_ctx verbs_alloc_ctx; /* Context for Verbs allocator. */ diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 24caf7e79..9c740f5e1 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -113,4 +113,14 @@ /* Maximum number of attempts to query link status before giving up. */ #define MLX5_MAX_LINK_QUERY_ATTEMPTS 5 +/* Reserved address space for UAR mapping. */ +#define MLX5_UAR_SIZE (1ULL << 32) + +/* Offset of reserved UAR address space to hugepage memory. Offset is used here + * to minimize possibility of address next to hugepage being used by other code + * in either primary or secondary process, failing to map TX UAR would make TX + * packets invisible to HW. + */ +#define MLX5_UAR_OFFSET (1ULL << 32) + #endif /* RTE_PMD_MLX5_DEFS_H_ */ diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index e35a6ab34..28761c299 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -204,7 +204,7 @@ struct mlx5_txq_data { volatile void *wqes; /* Work queue (use volatile to write into). */ volatile uint32_t *qp_db; /* Work queue doorbell. */ volatile uint32_t *cq_db; /* Completion queue doorbell. */ - volatile void *bf_reg; /* Blueflame register. */ + volatile void *bf_reg; /* Blueflame register remapped. */ struct mlx5_mr *mp2mr[MLX5_PMD_TX_MP_CACHE]; /* MR translation table. */ struct rte_mbuf *(*elts)[]; /* TX elements. */ struct mlx5_txq_stats stats; /* TX queue counters. */ @@ -229,6 +229,7 @@ struct mlx5_txq_ctrl { struct mlx5_txq_ibv *ibv; /* Verbs queue object. */ struct mlx5_txq_data txq; /* Data path structure. */ off_t uar_mmap_offset; /* UAR mmap offset for non-primary process. */ + volatile void *bf_reg_orig; /* Blueflame register from verbs. */ }; /* mlx5_rxq.c */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index d682ea2c9..da9494eb8 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -76,10 +76,13 @@ priv_txq_start(struct priv *priv) goto error; } } - return -ret; + ret = priv_tx_uar_remap(priv, priv->ctx->cmd_fd); + if (ret) + goto error; + return ret; error: priv_txq_stop(priv); - return -ret; + return ret; } static void diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 12f258bbb..f39b98a9a 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -217,7 +217,9 @@ mlx5_tx_queue_release(void *dpdk_txq) /** - * Map locally UAR used in Tx queues for BlueFlame doorbell. + * Mmap TX UAR(HW doorbell) pages into reserved UAR address space. + * Both primary and secondary process do mmap to make UAR address + * aligned. * * @param[in] priv * Pointer to private structure. @@ -234,11 +236,14 @@ priv_tx_uar_remap(struct priv *priv, int fd) uintptr_t pages[priv->txqs_n]; unsigned int pages_n = 0; uintptr_t uar_va; + uintptr_t off; void *addr; + void *ret; struct mlx5_txq_data *txq; struct mlx5_txq_ctrl *txq_ctrl; int already_mapped; size_t page_size = sysconf(_SC_PAGESIZE); + int r; memset(pages, 0, priv->txqs_n * sizeof(uintptr_t)); /* @@ -251,8 +256,10 @@ priv_tx_uar_remap(struct priv *priv, int fd) continue; txq = (*priv->txqs)[i]; txq_ctrl = container_of(txq, struct mlx5_txq_ctrl, txq); - uar_va = (uintptr_t)txq_ctrl->txq.bf_reg; - uar_va = RTE_ALIGN_FLOOR(uar_va, page_size); + /* UAR addr form verbs used to find dup and offset in page. */ + uar_va = (uintptr_t)txq_ctrl->bf_reg_orig; + off = uar_va & (page_size - 1); /* offset in page. */ + uar_va = RTE_ALIGN_FLOOR(uar_va, page_size); /* page addr. */ already_mapped = 0; for (j = 0; j != pages_n; ++j) { if (pages[j] == uar_va) { @@ -260,16 +267,30 @@ priv_tx_uar_remap(struct priv *priv, int fd) break; } } - if (already_mapped) - continue; - pages[pages_n++] = uar_va; - addr = mmap((void *)uar_va, page_size, - PROT_WRITE, MAP_FIXED | MAP_SHARED, fd, - txq_ctrl->uar_mmap_offset); - if (addr != (void *)uar_va) { - ERROR("call to mmap failed on UAR for txq %d\n", i); - return -1; + /* new address in reserved UAR address space. */ + addr = RTE_PTR_ADD(priv->uar_base, + uar_va & (MLX5_UAR_SIZE - 1)); + if (!already_mapped) { + pages[pages_n++] = uar_va; + /* fixed mmap to specified address in reserved + * address space. + */ + ret = mmap(addr, page_size, + PROT_WRITE, MAP_FIXED | MAP_SHARED, fd, + txq_ctrl->uar_mmap_offset); + if (ret != addr) { + /* fixed mmap have to return same address */ + ERROR("call to mmap failed on UAR for txq %d\n", + i); + r = ENXIO; + return r; + } } + if (rte_eal_process_type() == RTE_PROC_PRIMARY) /* save once */ + txq_ctrl->txq.bf_reg = RTE_PTR_ADD((void *)addr, off); + else + assert(txq_ctrl->txq.bf_reg == + RTE_PTR_ADD((void *)addr, off)); } return 0; } @@ -416,7 +437,7 @@ mlx5_priv_txq_ibv_new(struct priv *priv, uint16_t idx) txq_data->wqes = qp.sq.buf; txq_data->wqe_n = log2above(qp.sq.wqe_cnt); txq_data->qp_db = &qp.dbrec[MLX5_SND_DBR]; - txq_data->bf_reg = qp.bf.reg; + txq_ctrl->bf_reg_orig = qp.bf.reg; txq_data->cq_db = cq_info.dbrec; txq_data->cqes = (volatile struct mlx5_cqe (*)[]) @@ -717,6 +738,7 @@ mlx5_priv_txq_release(struct priv *priv, uint16_t idx) { unsigned int i; struct mlx5_txq_ctrl *txq; + size_t page_size = sysconf(_SC_PAGESIZE); if (!(*priv->txqs)[idx]) return 0; @@ -736,6 +758,9 @@ mlx5_priv_txq_release(struct priv *priv, uint16_t idx) txq->txq.mp2mr[i] = NULL; } } + if (priv->uar_base) + munmap((void *)RTE_ALIGN_FLOOR((uintptr_t)txq->txq.bf_reg, + page_size), page_size); if (rte_atomic32_dec_and_test(&txq->refcnt)) { txq_free_elts(txq); LIST_REMOVE(txq, next); -- 2.11.0