From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03AEDA0C4D; Mon, 8 Nov 2021 14:16:07 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E27D541144; Mon, 8 Nov 2021 14:16:06 +0100 (CET) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 0681841122 for ; Mon, 8 Nov 2021 14:16:04 +0100 (CET) X-IronPort-AV: E=McAfee;i="6200,9189,10161"; a="219120853" X-IronPort-AV: E=Sophos;i="5.87,218,1631602800"; d="scan'208";a="219120853" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2021 05:16:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,218,1631602800"; d="scan'208";a="451465453" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga006.jf.intel.com with ESMTP; 08 Nov 2021 05:16:03 -0800 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 8 Nov 2021 05:16:03 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Mon, 8 Nov 2021 05:16:03 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.176) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Mon, 8 Nov 2021 05:16:03 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lJTR2Ahv7Eb+vCuqd5tEPOsiMOJKdJOyxcpmUTwxM5RHxmkK5qrfRHk70UqfJ8dZ2D5rojuVFqdBRfAx01/E4Lz3ygMNn8bMW2U+ssWb4LKYl4TZL+P/AV7ecv49McrVsmjcDfE2jVVetWZHVbxpT1mE8ZLcWz9HCbwXsjB56ODY7x6PsqfRAlAS3qkchgO7ra2CB+Qtcug68xLKZpfsek9uxM7bkdd0zxEDRNrG6A8GFXRfy0v1DVZCYGDBe88+/JRnmh6bfBs1EtHZMVc8lBwewKICHGgGwxhoVMM+35exhKWJG2qFv3vWwaM7HNsNRIri3uCx/7j2VUX63hQ3WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pq6yzmcVzjRJJ0vaqXLNbdgCRyKhFIrvtm2CUElw1fk=; b=CVlsW/2pfP7Zi7WKAvzB8OvSMrgvHee6Z9De8zp8/5FtzTfNTE4BQZH/XykR5T5GWE92TZTE+dOD162QwH36M2na/xvI4BmiSxUvsajVe+ItFsTqUM0v4WE3JS6QoTuqVVi1nWGG8DIyCR7ezbWWxYGwB/GYiG+PbZB+ThhJ4MZXB9svGSiu5mTi+gB5Ys5k6Uyw17TWrDdDI2/HYwSw50cb/VpcrvslLqBsMUGcU0c+FQ+nwqxK1ljgZjaRl35ZBd8/C++fne02S8yw+QlHv4Hi0X2x87JTu/hZOKpQGFBix+TzSRdqFOHY/7xDo0vBOQdjc7MKzwVQEx2qpO3XnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pq6yzmcVzjRJJ0vaqXLNbdgCRyKhFIrvtm2CUElw1fk=; b=DtqfHhgXAqc1Smef9z+zT2FAlGv5wvLFxocFSPwX1dVggaXA+Nkqsh0648Oc/2NtkvuTH+ZdXDZDsd16D4ghljpGdarQY6prrq1NgkgCK/axVZkcCy+IcGCJHgajO53GTpIU6OodxWEnZ2o9DO5fRiOigTENrA4oqhHmE/i8URM= Authentication-Results: oss.nvidia.com; dkim=none (message not signed) header.d=none;oss.nvidia.com; dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by PH0PR11MB5173.namprd11.prod.outlook.com (2603:10b6:510:39::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10; Mon, 8 Nov 2021 13:16:01 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::bd7d:29be:3342:632c]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::bd7d:29be:3342:632c%7]) with mapi id 15.20.4669.016; Mon, 8 Nov 2021 13:16:01 +0000 Message-ID: <53a46630-2028-b298-12c2-553119327b75@intel.com> Date: Mon, 8 Nov 2021 13:15:56 +0000 Content-Language: en-US From: Ferruh Yigit To: Dmitry Kozlyuk , CC: Raslan Darawsheh , Matan Azrad , Viacheslav Ovsiienko References: <20211101095646.840095-1-dkozlyuk@nvidia.com> X-User: ferruhy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0088.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:190::21) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 Received: from [192.168.0.206] (37.228.236.146) by LO4P123CA0088.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:190::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 13:16:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: df783ff9-02a3-4ec2-c26c-08d9a2b9e91f X-MS-TrafficTypeDiagnostic: PH0PR11MB5173: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FgzPPuXj6lsdG5AeUUOfm/6Sb2tTPEbQkGpWj/ZF7+Tzn1hoeHC15ZGSYEEAoS13gdJhryWlFnbO+lK94afqHXxZtHLt1BemAziyvD/x9BbXIXD5cSR/f+fcCNZgoqLe6GC1w+LiugST9LCczE1Nl4Z35c5hERya9VFD96yLbU/DM5f5hYSCwCbhwdcvpdlxIQGObrZWuoEwFbco7j4nwQxfdfGTTCwuoEcepc50A48OjwYllskpCJCMJrZhzgNekQIcmYs92kmKbfT2r0/XqepE3bPtM3AAw6roBQK4JrI6k5N+xumEvTjpr94aYN37tCQ1rzagcm3GZlpmf6cq0L8VVraoR8s73QcXhBWAf4OGav3MBb00Lbj24PEA1u3u8ZZsDJ0+35NQ8s5+IIf6KnR5/dE6GDxHRc+pqomhFXSsJT3pfMaXE0Krhc4ZuyOg3rTgBoHBAlA7m08DWjwC6pXa8svMZ3ckYq8c193OCXM1HezxA7mkaW1QymBIvrZmRfr8KaNpX2cEUJX4BfWQNYEQkdy/GSnbzhjPITzwGA11SuzfTMhmsa4ej0qSu549ou25FVAYQTp4P8anCjdt6FEyu6ENS0krYF7WXFaoymY3cu1HyAoamtZMoTYgFPfVcSDyae7OeFnmie4SgecuN6prIKLHrgSLEVdSQdzexENwC+O/2ZdkPTw3vxKmcWq5oihH7dMNmY2AVTNYdoijsGqtZn47z8soQS8HO0pFBck= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(31696002)(38100700002)(36756003)(16576012)(8936002)(66946007)(83380400001)(31686004)(82960400001)(186003)(44832011)(53546011)(316002)(66476007)(5660300002)(54906003)(8676002)(2616005)(6666004)(508600001)(2906002)(26005)(86362001)(6486002)(66556008)(4326008)(956004)(83323001)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NHZXaE9MNnFNRkdPbWozcnpQMjU2dndTenViaGFCUjFySHBNTzJEQ2JEZ0Zh?= =?utf-8?B?dEMxT1BUWlpvTkZCVzNXUTJ6eDU4MHlkMVlPYVM1R3dEaTlZN2ZqeXpYUGt6?= =?utf-8?B?ZmROTXB5dXJvVUpveHJiZjkyRk1GZWFPTUROcGZQMms3TFVRY0s4SStJcHIz?= =?utf-8?B?OXh6OEFvYmVOaVlvUnpjUXJXSEt4U2VJTGFOVi96azQ4eDFTQzhmc3RMYmg3?= =?utf-8?B?K1lsTnR3dmt6VHV1am5xaUg1K3paVS85dnZCRzlCcTVMWnIzUTZERGJ1Nkdu?= =?utf-8?B?NTJaeWNWdHZZM05DL1dXWHRMcEYzbG8zZDBLb1JvYzFrWUsxWGtUenFqblkw?= =?utf-8?B?NGk4MG9janNGOGl3d1JZZ1dQeDEzcmFBb0x0dnYrUDJUUnk5NER3N1dCSDJ6?= =?utf-8?B?aUNZRm5ySE4yRXRpMy9JUzhES0pRelZsM3o2a3lQSWhQMWYrV3dYQ0g2QjdG?= =?utf-8?B?bDhIcmo0RDhSdXVtNVpTSEtXQUp2aDNNMWU5OTNyNlJoVE1UNnYzcHlOa3p6?= =?utf-8?B?N2VYdkk0ZUJGMndZNEFiSjZ2clhRUHRTM2cxYTBnZ1pIcDdKZkhWLzZjR0cw?= =?utf-8?B?SDNDWVp3TmdjNFduamJMOWRrMW1FWi9VS2E0ZmkrOHgvYmxFaDJRclVDYTBI?= =?utf-8?B?VVZnNzZITjZFbndDUzF6cHg3cmdHbHJmZXQzQTZxOWJxRnd5U3NRbVdUZHhY?= =?utf-8?B?bEcyTjA3SmI0Q1BGdDJlVWxTQzFSR2NiNDhROU1CZ1VlM2JQNXRlTVBjL3Fr?= =?utf-8?B?R2ROdHc1YUluVkt4ZWZIaUZFb0tXb1VoYURyM2srZi9yZmNjQWJzQVdMc1pW?= =?utf-8?B?S2VQZGE4REpjTzNDTjZWQWFHS2c0QWl6cFJtemo1RmVzelZuUHIzYVZmbjFa?= =?utf-8?B?WkpoR3RoSUd3MGUrdkt6YUhMc0RvaHFaOWZWMGZtcno1OThzWmxBdWdsL01P?= =?utf-8?B?ZWtHUmRuWDliVUt4VHBSaURoODBnUEIzK212R1M0eXMxdGpGTDRTTG1pcHhz?= =?utf-8?B?TWphZUxweDRkdnhuU1dZc1VZY2N4TkJEYysvd0JFWmxFQWZJVjQra0hsczRy?= =?utf-8?B?QlE4QS9tMm9wWlBJSkllQUtYOHFBbHEzL1ZvZW1LV1UweXNOTWEvSVFYZDMx?= =?utf-8?B?ZzR6SkxLdGptSlY4amhCTm1BNTB5YzVzemRyRk0rdVE0ZTJzUmVNZ1JvOURB?= =?utf-8?B?T2tyd3dpa1pyN3BaZWp3Z0xnY3JEcEdqRW03b1dxdWlxdUpQdWVUZ0dvQzNM?= =?utf-8?B?Z0VuM1pMSGpsdk42R3VYZyttWlZGcGwrWjhFUlFsY1A0aGVSOVBHclpJbU1o?= =?utf-8?B?OWNTSU1kQjQ5WEFLSVpNQWlLZFpScGhQZmtObEsrQjJOcE03WFVFOWwxK2FV?= =?utf-8?B?dmY0dWNoMk4yMjBIVXpJUGdtUHM5SzhwZnlpMVpua2g2aHIzVkd4b1RwMDFt?= =?utf-8?B?QTI4OVhDaUE5NW9TL001RFNVcnNQcTlBTTMzMTIxSXd6WGRvaWFJM3JXckRi?= =?utf-8?B?d0NLcms1Z1hqaHh0MUNvUTlQQktRRll2VVhUdWwweDEzRnA5NWJ1L2VSb0ZR?= =?utf-8?B?RzFoQUFjekVWTTNweldQTVdFMjZkcksxdE43dnFaNkJDMnVVMGtQRDdiTm5T?= =?utf-8?B?S2w0Lzg5OVMwdFIwS1pFV0g2R2ovTGhpNnpFZ2VUTk41K04wQmJCNDUxL21i?= =?utf-8?B?dStsMzdMMkIxUndnZXhsbmdrNjNYa0k3UzROeHcwMm5hQlRnb2ZaNDdMOFRn?= =?utf-8?B?UGMzdXRKTVdaM0tJSWtDaGd6UFJQWUhLL3ZTK3gxWkV4VEFCT3ZEdzY0QXBC?= =?utf-8?B?UlhrREtSZHN0MGFoM0ZIS01LQ2NPWVZsKy9zTS9lOVZXZFd3ZVlwd2MybjUv?= =?utf-8?B?ck9iWnRteWVBRjAzMTJBMURUNG5XRkxVYXozc2JKZTNyOFIwYlBTMmg1ZnVK?= =?utf-8?B?aUFmeFdlc0JZSlJxeEluZkJQck5wdkdpdWVoYTd6UC9kWHc1d0FrTjJoUjVm?= =?utf-8?B?T3VLK0o4NENsazZYejZFd2ZKVWtaYllianRTcnhqNXZGd2s0dm5jenl0SjBn?= =?utf-8?B?d1FOeG1oaHlYMHhKRW5xYnBBWHhiSS85TzlOT1I2OHkybS9URXpVTTRuNXFp?= =?utf-8?B?dXNKY3piL0pLeGFUK09FVmdrNEw5ZjF3Tkl4ekhyRUY2eHNacjRQOFpLSk5E?= =?utf-8?Q?dovXWFofIBpgkWmAGGL4XT8=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: df783ff9-02a3-4ec2-c26c-08d9a2b9e91f X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 13:16:01.5558 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z3xiv/AsUXbWBLEdbt4GF6INaBzYz/caNTNP8FOoqTj2/fHsaMjq04kOgi8ea6Wz/qeyzY+ekX8Yl9scmtQofg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5173 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] net/mlx5: fix split buffer Rx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 11/8/2021 1:10 PM, Ferruh Yigit wrote: > On 11/1/2021 9:56 AM, Dmitry Kozlyuk wrote: >> Routine to lookup LKey on Rx was assuming that the mbuf address >> always belongs to a single mempool: the one associated with an RxQ >> or the MPRQ mempool. This assumption is false for split buffers case. >> A wrong LKey was looked up, resulting in completion errors. >> Modify lookup routines to lookup LKey in the mbuf->pool >> for non-MPRQ cases both on Rx datapath and on queue initialization. >> >> Fixes: fec28ca0e3a9 ("net/mlx5: support mempool registration") >> >> Signed-off-by: Dmitry Kozlyuk >> Reviewed-by: Matan Azrad >> Reviewed-by: Viacheslav Ovsiienko >> --- >>   drivers/net/mlx5/mlx5_rx.c |  5 ++++- >>   drivers/net/mlx5/mlx5_rx.h | 37 +++++++++++++++++++++++++++++++++++-- >>   2 files changed, 39 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c >> index 258a645314..7296e81534 100644 >> --- a/drivers/net/mlx5/mlx5_rx.c >> +++ b/drivers/net/mlx5/mlx5_rx.c >> @@ -356,6 +356,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) >>           volatile struct mlx5_wqe_data_seg *scat; >>           uintptr_t addr; >>           uint32_t byte_count; >> +        uint32_t lkey; >>           if (mlx5_rxq_mprq_enabled(rxq)) { >>               struct mlx5_mprq_buf *buf = (*rxq->mprq_bufs)[i]; >> @@ -366,6 +367,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) >>                                1 << rxq->strd_num_n); >>               byte_count = (1 << rxq->strd_sz_n) * >>                       (1 << rxq->strd_num_n); >> +            lkey = mlx5_rx_addr2mr(rxq, addr); >>           } else { >>               struct rte_mbuf *buf = (*rxq->elts)[i]; >> @@ -373,13 +375,14 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) >>                       rxq->wqes)[i]; >>               addr = rte_pktmbuf_mtod(buf, uintptr_t); >>               byte_count = DATA_LEN(buf); >> +            lkey = mlx5_rx_mb2mr(rxq, buf); >>           } >>           /* scat->addr must be able to store a pointer. */ >>           MLX5_ASSERT(sizeof(scat->addr) >= sizeof(uintptr_t)); >>           *scat = (struct mlx5_wqe_data_seg){ >>               .addr = rte_cpu_to_be_64(addr), >>               .byte_count = rte_cpu_to_be_32(byte_count), >> -            .lkey = mlx5_rx_addr2mr(rxq, addr), >> +            .lkey = lkey, >>           }; >>       } >>       rxq->consumed_strd = 0; >> diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h >> index 4952fe1455..5903776e7a 100644 >> --- a/drivers/net/mlx5/mlx5_rx.h >> +++ b/drivers/net/mlx5/mlx5_rx.h >> @@ -266,7 +266,7 @@ uint16_t mlx5_rx_burst_mprq_vec(void *dpdk_rxq, struct rte_mbuf **pkts, >>   static int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq); >>   /** >> - * Query LKey from a packet buffer for Rx. No need to flush local caches >> + * Query LKey for an address on Rx. No need to flush local caches >>    * as the Rx mempool database entries are valid for the lifetime of the queue. >>    * >>    * @param rxq >> @@ -301,7 +301,40 @@ mlx5_rx_addr2mr(struct mlx5_rxq_data *rxq, uintptr_t addr) >>                        mr_ctrl, mp, addr); >>   } >> -#define mlx5_rx_mb2mr(rxq, mb) mlx5_rx_addr2mr(rxq, (uintptr_t)((mb)->buf_addr)) >> +/** >> + * Query LKey from a packet buffer for Rx. No need to flush local caches >> + * as the Rx mempool database entries are valid for the lifetime of the queue. >> + * >> + * @param rxq >> + *   Pointer to Rx queue structure. >> + * @param mb >> + *   Buffer to search the address of. >> + * >> + * @return >> + *   Searched LKey on success, UINT32_MAX on no match. >> + *   This function always succeeds on valid input. >> + */ >> +static __rte_always_inline uint32_t >> +mlx5_rx_mb2mr(struct mlx5_rxq_data *rxq, struct rte_mbuf *mb) >> +{ >> +    struct mlx5_mr_ctrl *mr_ctrl = &rxq->mr_ctrl; >> +    uintptr_t addr = (uintptr_t)mb->buf_addr; >> +    struct mlx5_rxq_ctrl *rxq_ctrl; >> +    uint32_t lkey; >> + >> +    /* Linear search on MR cache array. */ >> +    lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, >> +                   MLX5_MR_CACHE_N, addr); >> +    if (likely(lkey != UINT32_MAX)) >> +        return lkey; >> +    /* >> +     * Slower search in the mempool database on miss. >> +     * During queue creation rxq->sh is not yet set, so we use rxq_ctrl. >> +     */ >> +    rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq); >> +    return mlx5_mr_mempool2mr_bh(&rxq_ctrl->priv->sh->cdev->mr_scache, >> +                     mr_ctrl, mb->pool, addr); > > Hi Dmitry, > > I am getting build errors with this patch, although I can see CI build is passed, > can you please check if I am missing something, build error is: > > ../drivers/net/mlx5/mlx5_rx.h: In function ‘mlx5_rx_mb2mr’: > ../drivers/net/mlx5/mlx5_rx.h:354:47: error: ‘struct mlx5_rxq_ctrl’ has no member named ‘priv’ >   354 |         return mlx5_mr_mempool2mr_bh(&rxq_ctrl->priv->sh->cdev->mr_scache, >       |                                               ^~ > > My bad, Raslan pointed out that there is a v2 already, will continue with that one.