From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA0A4A054A for ; Wed, 15 Jun 2022 12:02:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E24304281A; Wed, 15 Jun 2022 12:02:51 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2085.outbound.protection.outlook.com [40.107.236.85]) by mails.dpdk.org (Postfix) with ESMTP id BC51740040; Wed, 15 Jun 2022 12:02:49 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YqoPCKSQ3VqiC+0eWuvLGbL6WTUvsADj1N7PWwIgml1r5hU5py9Sx+yRUkgEo91NhbbAuoEmuhyq24p8trC2BXZ2fWVL/DlxiBMtDst/0cZFaNClmoH3dAswk8RQd/4i/+d2GVPqMUp2D6EOtS7hDh49bGLtS43dq2YFWXMmhDGb6u9d+z2ML56iq6ytq7+1iPC37oiVhVjTaI79ZSc9xtrrxvOcI00fSDIXxitcMYPSHjfUEavy5600IlrKBF+rIW1GzZ6q5JqksTxSonZ0Mvf98hI+ijQALlvnQwQALB/71jJ8CYiOJsWBGguOyVVoi/+eRuyQL9JZTolY5FMuAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aSwgopt3X0Aag4vopR2aD2g69TQ2k5orHQwZnYB8xq8=; b=gEIgDSSwaspIEOjTlawAloqNBfNrjWb+VJ8ECjXc/4x+995tEStI96hZSWUdgCtgmdscZwDaGiQ/l24IRubGnrsy9Z7WxY8lFe8Wcev3crgpxxHQJvdPNGLwSki7Tg4m5aU7OTTu+e0xk5fwhA2TnXu1czVM8PAvry9Fvls9mVe5V0uwvuh1dGKmoG2xu/hFKKQR0Jo2qYY8XajVgVdQhfbgun3XXC8YhoK1GuR7PnesVIc8We6dmNXPjvW7TIGZ+T0YCABB2ZheHRqhFGHBoQy15jxqDW12DutN6jjvGHiltu3DityA7tuMvsNt7nlr87ai3DXabrHScNEj8ST5uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aSwgopt3X0Aag4vopR2aD2g69TQ2k5orHQwZnYB8xq8=; b=AZoQNRn8LXQRL+vScfnDAlpZHWDrHSTRerudhs1XPqxcQxGcdaQ5yOUZwiUwKEikGi91g8hEtDCKHfRlgCL8svYUlhMUWypA6Lx/aQt6AVeU2+ddx+tYDNyfELfKLAGgs8eoVr9M1ByLI/wgBf7NaQXLwDBBW8SXxQtzfwOlk4B2ZbuC0ZIGtPK8xDcUsVL7CRIbd46Dm+V3doCQGjbci6/yS3GGc8eG9ZDw2Uhl++QtmUCMy68+OYG1+o2gFUjzC8IoyWHVQpE0FCV6Tmp208jt+gyl2R+TzAEuCgraq5lezZr7tsjydEDb03PjaPAYqkOP00Ud3r8ew7A1RC0uKA== Received: from MW4PR04CA0309.namprd04.prod.outlook.com (2603:10b6:303:82::14) by MN2PR12MB3087.namprd12.prod.outlook.com (2603:10b6:208:d2::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13; Wed, 15 Jun 2022 10:02:48 +0000 Received: from CO1NAM11FT036.eop-nam11.prod.protection.outlook.com (2603:10b6:303:82:cafe::27) by MW4PR04CA0309.outlook.office365.com (2603:10b6:303:82::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13 via Frontend Transport; Wed, 15 Jun 2022 10:02:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT036.mail.protection.outlook.com (10.13.174.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5332.12 via Frontend Transport; Wed, 15 Jun 2022 10:02:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 15 Jun 2022 10:02:46 +0000 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Wed, 15 Jun 2022 03:02:44 -0700 From: Yajun Wu To: , CC: , , , , Subject: [PATCH v3] vdpa/mlx5: workaround var offset within page Date: Wed, 15 Jun 2022 13:02:27 +0300 Message-ID: <20220615100227.358768-1-yajunw@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220314014414.251869-1-yajunw@nvidia.com> References: <20220314014414.251869-1-yajunw@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9cf9ee38-761d-45b9-d323-08da4eb63323 X-MS-TrafficTypeDiagnostic: MN2PR12MB3087:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: koksnGAgr8sYy8Ay9HvKXkuS2C1nm3+T+tSeLF70xmMxk/Df6dqypTvvCWXo82C0EGJEog06fYYIWPGDMYOBK+N6kdXO1kCoe5AYa5HBtvAhYw3phIYSNfcOSqAw/S+7prIWz+eQuESkaRA44VHxB3G0XO8oVK2G1U7fXomHOf5WmA/DNRt3h1YPIldaIY/MHpysw/gVDu3ymDtrEwKgvKwyvV1/FlWfwdliQKozqs/gLe2ea0mUvbCKu0G0xQUKxzV/UJhtuhbME1g3EOce90948vWgAZyNS3by4vFYiz44e1F1U38HO27JEwiAIxQDxS6b2uV+3cQ/Tpukog+tVQpft0QnKwGBAKUEnYsCuSXRcRAvPxkzs8db5pbey8u0QWD3x2CQNEOpN2n61byxIr3jkI+9UNje0PvxkRlFn4hqQ4wJgLkkR4jyyAQFfqP9mpbkL2k9bUNOhqunWr5OuDJsWELFLzwwNqGQQhQK1LdQR2RBa06OLyW8sdyDIHJLnuN6FOzWFUXcKR0rbnkCx/lBZi46D8f+weraKhJwrdmp/b8Uf27PBxuwPkjEipL6iAYYomqzR02RJpKrApuqA84UyX9KfEls7CHGsaNzCZ2f0e8SPyrHe/YWAVpZdz58MLpuGcZ5NokJ+4RkiIAdL9sX5WafyRRDCp5qyr0ECPmex8qpi4amyRpTzt7Aa/6IIpGE0BIIHyadXCyfOkEWJ7PFCIz2XfxpeHmCh3qYsvQ= X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230016)(4636009)(36840700001)(40470700004)(46966006)(70586007)(54906003)(6286002)(356005)(26005)(82310400005)(5660300002)(186003)(2906002)(86362001)(1076003)(36860700001)(2616005)(110136005)(70206006)(8936002)(81166007)(8676002)(508600001)(316002)(6636002)(83380400001)(47076005)(55016003)(40460700003)(426003)(6666004)(16526019)(336012)(7696005)(4326008)(36756003)(2101003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2022 10:02:47.4409 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9cf9ee38-761d-45b9-d323-08da4eb63323 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT036.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3087 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org vDPA driver first uses kernel driver to allocate doorbell(VAR) area for each device. Then uses var->mmap_off and var->length to mmap uverbs device file as doorbell userspace virtual address. Current kernel driver provides var->mmap_off equal to page start of VAR. It's fine with x86 4K page server, because VAR physical address is only 4K aligned thus locate in 4K page start. But with aarch64 64K page server, the actual VAR physical address has offset within page(not locate in 64K page start). So vDPA driver need add this within page offset(caps.doorbell_bar_offset) to get right VAR virtual address. Fixes: 62c813706e4 ("vdpa/mlx5: map doorbell") Cc: stable@dpdk.org Signed-off-by: Yajun Wu Acked-by: Matan Azrad Reviewed-by: Maxime Coquelin --- drivers/vdpa/mlx5/mlx5_vdpa.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 76fa5d4299..8a33a0c9a1 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -560,6 +561,9 @@ mlx5_vdpa_create_dev_resources(struct mlx5_vdpa_priv *priv) rte_errno = errno; return -rte_errno; } + /* Add within page offset for 64K page system. */ + priv->virtq_db_addr = (char *)priv->virtq_db_addr + + ((rte_mem_page_size() - 1) & priv->caps.doorbell_bar_offset); DRV_LOG(DEBUG, "VAR address of doorbell mapping is %p.", priv->virtq_db_addr); priv->td = mlx5_devx_cmd_create_td(ctx); @@ -705,7 +709,9 @@ mlx5_vdpa_release_dev_resources(struct mlx5_vdpa_priv *priv) if (priv->td) claim_zero(mlx5_devx_cmd_destroy(priv->td)); if (priv->virtq_db_addr) - claim_zero(munmap(priv->virtq_db_addr, priv->var->length)); + /* Mask out the within page offset for munmap. */ + claim_zero(munmap((void *)((uintptr_t)priv->virtq_db_addr & + ~(rte_mem_page_size() - 1)), priv->var->length)); if (priv->var) mlx5_glue->dv_free_var(priv->var); } -- 2.30.2