From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-3.sys.kth.se (smtp-3.sys.kth.se [130.237.48.192]) by dpdk.org (Postfix) with ESMTP id BA1542C02 for ; Wed, 5 Sep 2018 11:00:43 +0200 (CEST) Received: from smtp-3.sys.kth.se (localhost.localdomain [127.0.0.1]) by smtp-3.sys.kth.se (Postfix) with ESMTP id 590BD41B7; Wed, 5 Sep 2018 11:00:43 +0200 (CEST) X-Virus-Scanned: by amavisd-new at kth.se Received: from smtp-3.sys.kth.se ([127.0.0.1]) by smtp-3.sys.kth.se (smtp-3.sys.kth.se [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZLOOpVTD0xia; Wed, 5 Sep 2018 11:00:36 +0200 (CEST) Received: from exdb01.ug.kth.se (unknown [192.168.32.111]) by smtp-3.sys.kth.se (Postfix) with ESMTPS id 8EAB7398E; Wed, 5 Sep 2018 11:00:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kth.se; s=default; t=1536138036; bh=89PU7PSrvB9nhW2MrbJImWCbAoC5Ku76nJTuYkeAnVI=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=lq7CW/tjoLvP84lr6LhoXHfDwGNiRqanLXIDSjdOuRlMLS8OdSDCXLLeHfZp2Rad/ KCfds2raI3VeFYYFkA6aI179Z1je2rH+rkbprEH/cDNlSVSmVCIaF9To4IeVuUOzfJ R270ViOjCgDZpriIey/2FEC4Q7sCZJ1eLPxoefBQ= Received: from exdb03.ug.kth.se (192.168.32.113) by exdb01.ug.kth.se (192.168.32.111) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 5 Sep 2018 11:00:03 +0200 Received: from exdb05.ug.kth.se (192.168.32.115) by exdb03.ug.kth.se (192.168.32.113) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 5 Sep 2018 11:00:03 +0200 Received: from exdb05.ug.kth.se ([192.168.32.115]) by exdb05.ug.kth.se ([192.168.32.115]) with mapi id 15.00.1367.000; Wed, 5 Sep 2018 11:00:03 +0200 From: Tom Barbette To: Shahaf Shuler , "dev@dpdk.org" , "Alex Rosenbaum" CC: Yongseok Koh , "john.mcnamara@intel.com" , "marko.kovacevic@intel.com" Thread-Topic: MLX5 should define the timestamp field in the doc Thread-Index: AQHURFrn477t1+S0lkKGIf1HTd0m9KThJ7twgAAZ7BeAABdtgIAAC5OC Date: Wed, 5 Sep 2018 09:00:03 +0000 Message-ID: <1536138003083.99523@kth.se> References: <1536071162684.44442@kth.se>, <1536131476329.46769@kth.se>, In-Reply-To: Accept-Language: fr-FR, sv-SE, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [130.237.202.15] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] MLX5 should define the timestamp field in the doc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2018 09:00:44 -0000 Actually I managed this patch to implement support for rte_eth_timesync_rea= d_time. Please tell me potential modifications, and if I shall submit it again as a= "normal" patch to dev ? --- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_ethdev.c | 30 ++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_glue.c | 8 ++++++++ drivers/net/mlx5/mlx5_glue.h | 2 ++ 5 files changed, 42 insertions(+) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c933e27..8c34794 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -324,6 +324,7 @@ const struct eth_dev_ops mlx5_dev_ops =3D { .xstats_reset =3D mlx5_xstats_reset, .xstats_get_names =3D mlx5_xstats_get_names, .dev_infos_get =3D mlx5_dev_infos_get, + .timesync_read_time =3D mlx5_timesync_read_time, .dev_supported_ptypes_get =3D mlx5_dev_supported_ptypes_get, .vlan_filter_set =3D mlx5_vlan_filter_set, .rx_queue_setup =3D mlx5_rx_queue_setup, diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 997b04a..5747304 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -217,6 +217,7 @@ int mlx5_set_flags(struct rte_eth_dev *dev, unsigned in= t keep, unsigned int flags); int mlx5_dev_configure(struct rte_eth_dev *dev); void mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *= info); +int mlx5_timesync_read_time(struct rte_eth_dev *dev, struct timespec* time= ); const uint32_t *mlx5_dev_supported_ptypes_get(struct rte_eth_dev *dev); int mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete); int mlx5_force_link_status_change(struct rte_eth_dev *dev, int status); diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.= c index 90488af..b7f0d91 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -480,6 +480,36 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte= _eth_dev_info *info) } /** + * Get device current time + * + * @param dev + * Pointer to Ethernet device structure. + * + * @param[out] time + * Time output value. + * + * @return + * 0 if the time has correctly been set + */ +int +mlx5_timesync_read_time(struct rte_eth_dev *dev, struct timespec *time) +{ + struct priv *priv =3D dev->data->dev_private; + struct ibv_values_ex values; + int err =3D 0; + + values.comp_mask =3D IBV_VALUES_MASK_RAW_CLOCK; + if ((err =3D mlx5_glue->query_rt_values_ex(priv->ctx, &values)) !=3D 0= ) { + DRV_LOG(WARNING, "Could not query time !"); + return err; + } + + *time =3D values.raw_clock; + return 0; +} + + +/** * Get supported packet types. * * @param dev diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index c7965e5..3c72f5b 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -84,6 +84,13 @@ mlx5_glue_query_device_ex(struct ibv_context *context, } static int +mlx5_glue_query_rt_values_ex(struct ibv_context *context, + struct ibv_values_ex* values) +{ + return ibv_query_rt_values_ex(context, values); +} + +static int mlx5_glue_query_port(struct ibv_context *context, uint8_t port_num, struct ibv_port_attr *port_attr) { @@ -354,6 +361,7 @@ const struct mlx5_glue *mlx5_glue =3D &(const struct ml= x5_glue){ .close_device =3D mlx5_glue_close_device, .query_device =3D mlx5_glue_query_device, .query_device_ex =3D mlx5_glue_query_device_ex, + .query_rt_values_ex =3D mlx5_glue_query_rt_values_ex, .query_port =3D mlx5_glue_query_port, .create_comp_channel =3D mlx5_glue_create_comp_channel, .destroy_comp_channel =3D mlx5_glue_destroy_comp_channel, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index e584d36..0582e95 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -54,6 +54,8 @@ struct mlx5_glue { int (*query_device_ex)(struct ibv_context *context, const struct ibv_query_device_ex_input *input, struct ibv_device_attr_ex *attr); + int (*query_rt_values_ex)(struct ibv_context *context, + struct ibv_values_ex *values); int (*query_port)(struct ibv_context *context, uint8_t port_num, struct ibv_port_attr *port_attr); struct ibv_comp_channel *(*create_comp_channel) -- 2.7.4 ________________________________ De : Shahaf Shuler Envoy=E9 : mercredi 5 septembre 2018 10:18 =C0 : Tom Barbette; dev@dpdk.org; Alex Rosenbaum Cc : Yongseok Koh; john.mcnamara@intel.com; marko.kovacevic@intel.com Objet : RE: MLX5 should define the timestamp field in the doc Thanks for the details. The use case is clear. We will take it internally to see when we can suppor= t it. AFAIK we cannot read the internal time from userspace. Adding also AlexR to comment From: Tom Barbette Sent: Wednesday, September 5, 2018 10:11 AM To: Shahaf Shuler ; dev@dpdk.org Cc: Yongseok Koh ; john.mcnamara@intel.com; marko.kovac= evic@intel.com Subject: RE: MLX5 should define the timestamp field in the doc Thanks for your answer Shahaf ! We're trying to measure the latency of packets going through various servic= e chains inside individual "server". Eg. we can see that on Server 1, the = latency for the service chain handling HTTP packets is ~800ns (+ max and mi= ns, tail latency, etc). What we do now is to timestamp packets right after = they are received, and compute the difference with the timestamp just befor= e they are sent. Over a cluster this shows us where the latency is happenin= g. We would like this "box" latency to include the time spent in queues, and f= or that the hardware timestamp seems fit-for-purpose as it would timestamp = the packets before the software queues. Moreover, as we use batching, we lo= se a lot of precision as we timestamp a whole batch at once. I'm pretty sure this use case is of interest for many others. Tail latency = is of the essence nowadays, and finding where packets get delayed precisely= is important. ? Instead of converting the timestamp to real time, in this very use case it = seems the Mellanox card could actually be our unique source of time, we jus= t need to be able to convert ticks to seconds. Any chance we can run an equivalent of mlx5_read_internal_timer (https://el= ixir.bootlin.com/linux/v4.18.5/source/drivers/net/ethernet/mellanox/mlx5/co= re/main.c#L623) ?from userspace ? Are these registers also mapped, or can b= e done so with a few changes? With only that we can actually derive the fre= quency and the offset easily.? Tom