From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6220A0032; Sat, 18 Jun 2022 11:03:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A95F42B6E; Sat, 18 Jun 2022 11:03:37 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2056.outbound.protection.outlook.com [40.107.243.56]) by mails.dpdk.org (Postfix) with ESMTP id 016794281A for ; Sat, 18 Jun 2022 11:03:36 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mlbWPaaE879JgNjKjhiMEo8Lt4SD/BbOIC1fsWP09GTsfR9smPkpzXAP0yrbseKWBtm5m7PvWPLSh2dV9HZL8EF9rCuymGILpnpPvWJ7t0ICzFLhsYzbetgmH/f4RUwjFZWpEyoW80texQIgvF5JChbVnJ+tyhqwWKfxweTiuUJk8VJOi7Fe+HWyjR3NAk7mcBptHOWG7w7++z4Sr4rywpAu2i7dzLRs3u74jJlr76ETt7gSm9gpqooeK79jL4x4YxsLgv8as/01kAcxkdyVZVZp9bKYPmh0+LKfN/6C96XOpX7umBmFUcz3NHxIN7+ljGim+1NW2yyrgXKb5XRuPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Jf0WGDAUT4VEUrzgtuXTuMFdmDb62J2MRM4/YqXo9ik=; b=eBxRds8jD7CDZs/vbyzJTa8piN2cjPs1USb0hiK4TuGQdtKSahuW72RUluTXrvMc1xDVb2pnAK3r1SfOC/A2k6vDT9AXD+Oeh2PvWwJQKpbnavN3DPujKlGfXwBcTjklbNOcI0KxF1JBCd4IhshV9olyfECJeOX+0HeUUJeHP4aDokbpzJjCTR/YTd4ODhZg4mmMjj3z7iCgfE4utp5cyJ/wV7OgVq/NC/TaJvV3tmDUnyiXtdUJaOBO4l+KrfKR8e6vrJur3EyfCxeWPTWiidevMG9wCHnQTPNFPLVpMqJMXr+keNfuoaalJ9wPMKoAPiVrbdhWeSaENdxH+aFhag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Jf0WGDAUT4VEUrzgtuXTuMFdmDb62J2MRM4/YqXo9ik=; b=hsTB5GbNpyQ9UwbYVXrfetPK3NOWIXHyECeA+vOlFMPnYKt8JunpCsdHfjz/3NeEzVC/2ZJJbTaH48swZIbgnOvduCCYRf6t3OYR92ltgPOahLjKH240A/XX9vC/m46oomlWwHwVA6OyprRItRGtDGDd0o4eSAKxvz9FMoqgSnNAfftKHKId0Wware7WMvcrHkaWQwUDt77M10DSVp4v46uJPv+WGGSMoH3lIt+J/ZBpCfeuIRNCQ+UdsuEWXvGU/fWhV4WFrtoMCWVXfEwMZvtY2zsf1Ww2rj3gngbb8avPBSek4toyFOX+h23pkLl3ecJWKUDwU6/1wnZaBC7Dnw== Received: from BN6PR1401CA0012.namprd14.prod.outlook.com (2603:10b6:405:4b::22) by BYAPR12MB3511.namprd12.prod.outlook.com (2603:10b6:a03:132::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.14; Sat, 18 Jun 2022 09:03:32 +0000 Received: from BN8NAM11FT047.eop-nam11.prod.protection.outlook.com (2603:10b6:405:4b:cafe::f7) by BN6PR1401CA0012.outlook.office365.com (2603:10b6:405:4b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.15 via Frontend Transport; Sat, 18 Jun 2022 09:03:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT047.mail.protection.outlook.com (10.13.177.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5353.14 via Frontend Transport; Sat, 18 Jun 2022 09:03:32 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sat, 18 Jun 2022 09:03:31 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sat, 18 Jun 2022 02:03:28 -0700 From: Li Zhang To: , , , CC: , , , , Subject: [PATCH v4 05/15] common/mlx5: extend virtq modifiable fields Date: Sat, 18 Jun 2022 12:02:48 +0300 Message-ID: <20220618090258.91157-6-lizh@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220618090258.91157-1-lizh@nvidia.com> References: <20220408075606.33056-1-lizh@nvidia.com> <20220618090258.91157-1-lizh@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e3246ade-87cb-4184-159b-08da51096b6c X-MS-TrafficTypeDiagnostic: BYAPR12MB3511:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OJKWtNwGzLvw+oh6XKCDTl5v1LWUIgp0OEOhz26hUokSzj9/DFO2rKN5vBNFcAYSuQ9HH+wtZom/rMv9ryNZeGImFovWmbNWsPACc1oyaySJIiypvDAVuLov0TlMu/ehFZchhBJ2vNzbTGB4z+PmdKtBlf42lojsG/0jGu/+GXyEZVvv/OFr8P0AM9hwOCiRYquHiKB+oomTCpDkGLz8HRVw9GMPs4DoiAMw+2YQo1/R5U5V+A7spILJSP1wI4GtdLc4Jps1XK8jq3OjCVCVOLbvEUeG6LDN08ixnVh9qGpwcpM1EZ8AjB5gn7wZruA7Ur+8TZud/gvhN/1fu0wMGNrgJ4l+r+ltFmB2y6g7vj5ukaFcdDZNAirwcQR01pDInM0JzNfMnWlh1ZwNfe04vbCqRcRa0biTsgskPRljZeYB28WlDs1vR5PaWNC+d/ZaHVoCs5KUUcpse5vWKSBpe7MeaHPIvBehdXPNkRYPbFAfcI1i//yBBMycle3adZst4gAcmp/ai4OSSktjK6ZM0COpC/ZCMmIOTyHw7UMC3oqqe2BPi7BvkBQIMC0QKNe0mr9yC2Wm62FQRt3Oc56KdjnGKm6DT3c9bcWy91XVQD0WxMynzeHDPOBWPr8SryJs0rOv6bvf1AIlFmLoldjIb/yc+Hr2m6KvAg0/5sVkT79O+E3h2nYJwcAnDIwGQoTEu2plup/hqLVleD0LzpGKxw== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230016)(4636009)(40470700004)(36840700001)(46966006)(5660300002)(2616005)(81166007)(6286002)(336012)(316002)(6666004)(70586007)(356005)(4326008)(70206006)(7696005)(8676002)(36756003)(86362001)(82310400005)(40460700003)(55016003)(2906002)(26005)(47076005)(186003)(110136005)(54906003)(426003)(6636002)(36860700001)(8936002)(1076003)(83380400001)(16526019)(498600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jun 2022 09:03:32.3428 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e3246ade-87cb-4184-159b-08da51096b6c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT047.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3511 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org A virtq configuration can be modified after the virtq creation. Added the following modifiable fields: 1.address fields: desc_addr/used_addr/available_addr 2.hw_available_index 3.hw_used_index 4.virtio_q_type 5.version type 6.queue mkey 7.feature bit mask: tso_ipv4/tso_ipv6/tx_csum/rx_csum 8.event mode: event_mode/event_qpn_or_msix Signed-off-by: Li Zhang Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 70 +++++++++++++++++++++++----- drivers/common/mlx5/mlx5_devx_cmds.h | 6 ++- drivers/common/mlx5/mlx5_prm.h | 13 +++++- 3 files changed, 76 insertions(+), 13 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 1d6d6578d6..1b68c37092 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -545,6 +545,15 @@ mlx5_devx_cmd_query_hca_vdpa_attr(void *ctx, vdpa_attr->log_doorbell_stride = MLX5_GET(virtio_emulation_cap, hcattr, log_doorbell_stride); + vdpa_attr->vnet_modify_ext = + MLX5_GET(virtio_emulation_cap, hcattr, + vnet_modify_ext); + vdpa_attr->virtio_net_q_addr_modify = + MLX5_GET(virtio_emulation_cap, hcattr, + virtio_net_q_addr_modify); + vdpa_attr->virtio_q_index_modify = + MLX5_GET(virtio_emulation_cap, hcattr, + virtio_q_index_modify); vdpa_attr->log_doorbell_bar_size = MLX5_GET(virtio_emulation_cap, hcattr, log_doorbell_bar_size); @@ -2074,27 +2083,66 @@ mlx5_devx_cmd_modify_virtq(struct mlx5_devx_obj *virtq_obj, MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type, MLX5_GENERAL_OBJ_TYPE_VIRTQ); MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_id, virtq_obj->id); - MLX5_SET64(virtio_net_q, virtq, modify_field_select, attr->type); + MLX5_SET64(virtio_net_q, virtq, modify_field_select, + attr->mod_fields_bitmap); MLX5_SET16(virtio_q, virtctx, queue_index, attr->queue_index); - switch (attr->type) { - case MLX5_VIRTQ_MODIFY_TYPE_STATE: + if (!attr->mod_fields_bitmap) { + DRV_LOG(ERR, "Failed to modify VIRTQ for no type set."); + rte_errno = EINVAL; + return -rte_errno; + } + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_STATE) MLX5_SET16(virtio_net_q, virtq, state, attr->state); - break; - case MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_PARAMS: + if (attr->mod_fields_bitmap & + MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_PARAMS) { MLX5_SET(virtio_net_q, virtq, dirty_bitmap_mkey, attr->dirty_bitmap_mkey); MLX5_SET64(virtio_net_q, virtq, dirty_bitmap_addr, attr->dirty_bitmap_addr); MLX5_SET(virtio_net_q, virtq, dirty_bitmap_size, attr->dirty_bitmap_size); - break; - case MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_DUMP_ENABLE: + } + if (attr->mod_fields_bitmap & + MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_DUMP_ENABLE) MLX5_SET(virtio_net_q, virtq, dirty_bitmap_dump_enable, attr->dirty_bitmap_dump_enable); - break; - default: - rte_errno = EINVAL; - return -rte_errno; + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_QUEUE_PERIOD) { + MLX5_SET(virtio_q, virtctx, queue_period_mode, + attr->hw_latency_mode); + MLX5_SET(virtio_q, virtctx, queue_period_us, + attr->hw_max_latency_us); + MLX5_SET(virtio_q, virtctx, queue_max_count, + attr->hw_max_pending_comp); + } + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_ADDR) { + MLX5_SET64(virtio_q, virtctx, desc_addr, attr->desc_addr); + MLX5_SET64(virtio_q, virtctx, used_addr, attr->used_addr); + MLX5_SET64(virtio_q, virtctx, available_addr, + attr->available_addr); + } + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_HW_AVAILABLE_INDEX) + MLX5_SET16(virtio_net_q, virtq, hw_available_index, + attr->hw_available_index); + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_HW_USED_INDEX) + MLX5_SET16(virtio_net_q, virtq, hw_used_index, + attr->hw_used_index); + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_Q_TYPE) + MLX5_SET16(virtio_q, virtctx, virtio_q_type, attr->q_type); + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_VERSION_1_0) + MLX5_SET16(virtio_q, virtctx, virtio_version_1_0, + attr->virtio_version_1_0); + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_Q_MKEY) + MLX5_SET(virtio_q, virtctx, virtio_q_mkey, attr->mkey); + if (attr->mod_fields_bitmap & + MLX5_VIRTQ_MODIFY_TYPE_QUEUE_FEATURE_BIT_MASK) { + MLX5_SET16(virtio_net_q, virtq, tso_ipv4, attr->tso_ipv4); + MLX5_SET16(virtio_net_q, virtq, tso_ipv6, attr->tso_ipv6); + MLX5_SET16(virtio_net_q, virtq, tx_csum, attr->tx_csum); + MLX5_SET16(virtio_net_q, virtq, rx_csum, attr->rx_csum); + } + if (attr->mod_fields_bitmap & MLX5_VIRTQ_MODIFY_TYPE_EVENT_MODE) { + MLX5_SET16(virtio_q, virtctx, event_mode, attr->event_mode); + MLX5_SET(virtio_q, virtctx, event_qpn_or_msix, attr->qp_id); } ret = mlx5_glue->devx_obj_modify(virtq_obj->obj, in, sizeof(in), out, sizeof(out)); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 3747ef9e33..ec6467d927 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -74,6 +74,9 @@ struct mlx5_hca_vdpa_attr { uint32_t log_doorbell_stride:5; uint32_t log_doorbell_bar_size:5; uint32_t queue_counters_valid:1; + uint32_t vnet_modify_ext:1; + uint32_t virtio_net_q_addr_modify:1; + uint32_t virtio_q_index_modify:1; uint32_t max_num_virtio_queues; struct { uint32_t a; @@ -465,7 +468,7 @@ struct mlx5_devx_virtq_attr { uint32_t tis_id; uint32_t counters_obj_id; uint64_t dirty_bitmap_addr; - uint64_t type; + uint64_t mod_fields_bitmap; uint64_t desc_addr; uint64_t used_addr; uint64_t available_addr; @@ -475,6 +478,7 @@ struct mlx5_devx_virtq_attr { uint64_t offset; } umems[3]; uint8_t error_type; + uint8_t q_type; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 8a2f55c33e..5f58a6ee1d 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1802,7 +1802,9 @@ struct mlx5_ifc_virtio_emulation_cap_bits { u8 virtio_queue_type[0x8]; u8 reserved_at_20[0x13]; u8 log_doorbell_stride[0x5]; - u8 reserved_at_3b[0x3]; + u8 vnet_modify_ext[0x1]; + u8 virtio_net_q_addr_modify[0x1]; + u8 virtio_q_index_modify[0x1]; u8 log_doorbell_bar_size[0x5]; u8 doorbell_bar_offset[0x40]; u8 reserved_at_80[0x8]; @@ -3024,6 +3026,15 @@ enum { MLX5_VIRTQ_MODIFY_TYPE_STATE = (1UL << 0), MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_PARAMS = (1UL << 3), MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_DUMP_ENABLE = (1UL << 4), + MLX5_VIRTQ_MODIFY_TYPE_QUEUE_PERIOD = (1UL << 5), + MLX5_VIRTQ_MODIFY_TYPE_ADDR = (1UL << 6), + MLX5_VIRTQ_MODIFY_TYPE_HW_AVAILABLE_INDEX = (1UL << 7), + MLX5_VIRTQ_MODIFY_TYPE_HW_USED_INDEX = (1UL << 8), + MLX5_VIRTQ_MODIFY_TYPE_Q_TYPE = (1UL << 9), + MLX5_VIRTQ_MODIFY_TYPE_VERSION_1_0 = (1UL << 10), + MLX5_VIRTQ_MODIFY_TYPE_Q_MKEY = (1UL << 11), + MLX5_VIRTQ_MODIFY_TYPE_QUEUE_FEATURE_BIT_MASK = (1UL << 12), + MLX5_VIRTQ_MODIFY_TYPE_EVENT_MODE = (1UL << 13), }; struct mlx5_ifc_virtio_q_bits { -- 2.31.1