From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22991A0549; Thu, 16 Jul 2020 10:27:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 080D71BFD1; Thu, 16 Jul 2020 10:23:56 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id A28291BF83 for ; Thu, 16 Jul 2020 10:23:40 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with SMTP; 16 Jul 2020 11:23:36 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06G8Naoa029045; Thu, 16 Jul 2020 11:23:36 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 06G8NZlg006767; Thu, 16 Jul 2020 08:23:35 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 06G8NZD4006766; Thu, 16 Jul 2020 08:23:35 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, olivier.matz@6wind.com, thomas@monjalon.net, ferruh.yigit@intel.com Date: Thu, 16 Jul 2020 08:23:20 +0000 Message-Id: <1594887800-6563-18-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594887800-6563-1-git-send-email-viacheslavo@mellanox.com> References: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> <1594887800-6563-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v3 17/17] net/mlx5: convert Rx timestamps in realtime format X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The ConnectX-6DX supports the timestamps in various formats, the new realtime format is introduced - the upper 32-bit word of timestamp contains the UTC seconds and the lower 32-bit word contains the nanoseconds. This patch detects what format is configured in the NIC and performs the conversion accordingly. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/linux/mlx5_os.c | 21 +++++++++++++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_rxq.c | 24 ++++++++++++++ drivers/net/mlx5/mlx5_rxtx.c | 6 +++- drivers/net/mlx5/mlx5_rxtx.h | 4 +++ drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 34 +++++++++++++++----- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 54 ++++++++++++++++++++++---------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 34 +++++++++++++++----- drivers/net/mlx5/mlx5_trigger.c | 6 ++-- 9 files changed, 149 insertions(+), 35 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index e7241d8..f228bab 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -936,6 +936,27 @@ goto error; #endif } + if (config.devx) { + uint32_t reg[MLX5_ST_SZ_DW(register_mtutc)]; + + err = mlx5_devx_cmd_register_read + (sh->ctx, MLX5_REGISTER_ID_MTUTC, 0, + reg, MLX5_ST_SZ_DW(register_mtutc)); + if (!err) { + uint32_t ts_mode; + + /* MTUTC register is read successfully. */ + ts_mode = MLX5_GET(register_mtutc, reg, + time_stamp_mode); + if (ts_mode == MLX5_MTUTC_TIMESTAMP_MODE_REAL_TIME) + config.rt_timestamp = 1; + } else { + /* Kernel does not support register reading. */ + if (config.hca_attr.dev_freq_khz == + (NS_PER_S / MS_PER_S)) + config.rt_timestamp = 1; + } + } if (config.mprq.enabled && mprq) { if (config.mprq.stride_num_n && (config.mprq.stride_num_n > mprq_max_stride_num_n || diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 6e4f055..1aa3a3e 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -216,6 +216,7 @@ struct mlx5_dev_config { unsigned int devx:1; /* Whether devx interface is available or not. */ unsigned int dest_tir:1; /* Whether advanced DR API is available. */ unsigned int reclaim_mode:2; /* Memory reclaim mode. */ + unsigned int rt_timestamp:1; /* realtime timestamp format. */ struct { unsigned int enabled:1; /* Whether MPRQ is enabled. */ unsigned int stride_num_n; /* Number of strides. */ diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 2681322..7dd06e8 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -2972,3 +2972,27 @@ struct mlx5_hrxq * priv->drop_queue.hrxq = NULL; } } + + +/** + * Set the Rx queue timestamp conversion parameters + * + * @param[in] dev + * Pointer to the Ethernet device structure. + */ +void +mlx5_rxq_timestamp_set(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_ctx_shared *sh = priv->sh; + struct mlx5_rxq_data *data; + unsigned int i; + + for (i = 0; i != priv->rxqs_n; ++i) { + if (!(*priv->rxqs)[i]) + continue; + data = (*priv->rxqs)[i]; + data->sh = sh; + data->rt_timestamp = priv->config.rt_timestamp; + } +} diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index e511142..65239f9 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1389,7 +1389,11 @@ enum mlx5_txcmp_code { pkt->vlan_tci = rte_be_to_cpu_16(cqe->vlan_info); } if (rxq->hw_timestamp) { - pkt->timestamp = rte_be_to_cpu_64(cqe->timestamp); + uint64_t ts = rte_be_to_cpu_64(cqe->timestamp); + + if (rxq->rt_timestamp) + ts = mlx5_txpp_convert_rx_ts(rxq->sh, ts); + pkt->timestamp = ts; pkt->ol_flags |= PKT_RX_TIMESTAMP; } } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index d082cd7..5116a15 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -109,6 +109,7 @@ enum mlx5_rxq_err_state { struct mlx5_rxq_data { unsigned int csum:1; /* Enable checksum offloading. */ unsigned int hw_timestamp:1; /* Enable HW timestamp. */ + unsigned int rt_timestamp:1; /* Realtime timestamp format. */ unsigned int vlan_strip:1; /* Enable VLAN stripping. */ unsigned int crc_present:1; /* CRC must be subtracted. */ unsigned int sges_n:3; /* Log 2 of SGEs (max buffers per packet). */ @@ -148,6 +149,7 @@ struct mlx5_rxq_data { struct rte_mempool *mp; struct rte_mempool *mprq_mp; /* Mempool for Multi-Packet RQ. */ struct mlx5_mprq_buf *mprq_repl; /* Stashed mbuf for replenish. */ + struct mlx5_dev_ctx_shared *sh; /* Shared context. */ uint16_t idx; /* Queue index. */ struct mlx5_rxq_stats stats; rte_xmm_t mbuf_initializer; /* Default rearm/flags for vectorized Rx. */ @@ -442,6 +444,8 @@ uint32_t mlx5_hrxq_get(struct rte_eth_dev *dev, void mlx5_hrxq_drop_release(struct rte_eth_dev *dev); uint64_t mlx5_get_rx_port_offloads(void); uint64_t mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev); +void mlx5_rxq_timestamp_set(struct rte_eth_dev *dev); + /* mlx5_txq.c */ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h index b55138a..f5414ee 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h @@ -1024,14 +1024,32 @@ /* D.5 fill in mbuf - rearm_data and packet_type. */ rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); if (rxq->hw_timestamp) { - pkts[pos]->timestamp = - rte_be_to_cpu_64(cq[pos].timestamp); - pkts[pos + 1]->timestamp = - rte_be_to_cpu_64(cq[pos + p1].timestamp); - pkts[pos + 2]->timestamp = - rte_be_to_cpu_64(cq[pos + p2].timestamp); - pkts[pos + 3]->timestamp = - rte_be_to_cpu_64(cq[pos + p3].timestamp); + if (rxq->rt_timestamp) { + struct mlx5_dev_ctx_shared *sh = rxq->sh; + uint64_t ts; + + ts = rte_be_to_cpu_64(cq[pos].timestamp); + pkts[pos]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p1].timestamp); + pkts[pos + 1]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p2].timestamp); + pkts[pos + 2]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p3].timestamp); + pkts[pos + 3]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + } else { + pkts[pos]->timestamp = rte_be_to_cpu_64 + (cq[pos].timestamp); + pkts[pos + 1]->timestamp = rte_be_to_cpu_64 + (cq[pos + p1].timestamp); + pkts[pos + 2]->timestamp = rte_be_to_cpu_64 + (cq[pos + p2].timestamp); + pkts[pos + 3]->timestamp = rte_be_to_cpu_64 + (cq[pos + p3].timestamp); + } } if (rxq->dynf_meta) { uint64_t flag = rxq->flow_meta_mask; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 3007c03..555c342 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -694,22 +694,44 @@ rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag, opcode, &elts[pos]); if (rxq->hw_timestamp) { - elts[pos]->timestamp = - rte_be_to_cpu_64( - container_of(p0, struct mlx5_cqe, - pkt_info)->timestamp); - elts[pos + 1]->timestamp = - rte_be_to_cpu_64( - container_of(p1, struct mlx5_cqe, - pkt_info)->timestamp); - elts[pos + 2]->timestamp = - rte_be_to_cpu_64( - container_of(p2, struct mlx5_cqe, - pkt_info)->timestamp); - elts[pos + 3]->timestamp = - rte_be_to_cpu_64( - container_of(p3, struct mlx5_cqe, - pkt_info)->timestamp); + if (rxq->rt_timestamp) { + struct mlx5_dev_ctx_shared *sh = rxq->sh; + uint64_t ts; + + ts = rte_be_to_cpu_64 + (container_of(p0, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64 + (container_of(p1, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 1]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64 + (container_of(p2, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 2]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64 + (container_of(p3, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 3]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + } else { + elts[pos]->timestamp = rte_be_to_cpu_64 + (container_of(p0, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 1]->timestamp = rte_be_to_cpu_64 + (container_of(p1, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 2]->timestamp = rte_be_to_cpu_64 + (container_of(p2, struct mlx5_cqe, + pkt_info)->timestamp); + elts[pos + 3]->timestamp = rte_be_to_cpu_64 + (container_of(p3, struct mlx5_cqe, + pkt_info)->timestamp); + } } if (!!rxq->flow_meta_mask) { /* This code is subject for futher optimization. */ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index da5960a..34e3397 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -655,14 +655,32 @@ /* D.5 fill in mbuf - rearm_data and packet_type. */ rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); if (rxq->hw_timestamp) { - pkts[pos]->timestamp = - rte_be_to_cpu_64(cq[pos].timestamp); - pkts[pos + 1]->timestamp = - rte_be_to_cpu_64(cq[pos + p1].timestamp); - pkts[pos + 2]->timestamp = - rte_be_to_cpu_64(cq[pos + p2].timestamp); - pkts[pos + 3]->timestamp = - rte_be_to_cpu_64(cq[pos + p3].timestamp); + if (rxq->rt_timestamp) { + struct mlx5_dev_ctx_shared *sh = rxq->sh; + uint64_t ts; + + ts = rte_be_to_cpu_64(cq[pos].timestamp); + pkts[pos]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p1].timestamp); + pkts[pos + 1]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p2].timestamp); + pkts[pos + 2]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + ts = rte_be_to_cpu_64(cq[pos + p3].timestamp); + pkts[pos + 3]->timestamp = + mlx5_txpp_convert_rx_ts(sh, ts); + } else { + pkts[pos]->timestamp = rte_be_to_cpu_64 + (cq[pos].timestamp); + pkts[pos + 1]->timestamp = rte_be_to_cpu_64 + (cq[pos + p1].timestamp); + pkts[pos + 2]->timestamp = rte_be_to_cpu_64 + (cq[pos + p2].timestamp); + pkts[pos + 3]->timestamp = rte_be_to_cpu_64 + (cq[pos + p3].timestamp); + } } if (rxq->dynf_meta) { /* This code is subject for futher optimization. */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 29aef54..6e5a730 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -329,9 +329,11 @@ dev->data->port_id); goto error; } - /* Set a mask and offset of dynamic metadata flows into Rx queues*/ + /* Set a mask and offset of dynamic metadata flows into Rx queues. */ mlx5_flow_rxq_dynf_metadata_set(dev); - /* Set a mask and offset of scheduling on timestamp into Tx queues*/ + /* Set flags and context to convert Rx timestamps. */ + mlx5_rxq_timestamp_set(dev); + /* Set a mask and offset of scheduling on timestamp into Tx queues. */ mlx5_txq_dynf_timestamp_set(dev); /* * In non-cached mode, it only needs to start the default mreg copy -- 1.8.3.1