From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19BF2A0350; Fri, 26 Jun 2020 08:41:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2E1031B3BB; Fri, 26 Jun 2020 08:41:54 +0200 (CEST) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id ACE771B203 for ; Fri, 26 Jun 2020 08:41:52 +0200 (CEST) Received: by mail-wr1-f66.google.com with SMTP id z13so8330596wrw.5 for ; Thu, 25 Jun 2020 23:41:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=EVXXrzlGkfZYargojUNypjwD4u+3zFdZxO0ELZcNMNc=; b=TQzw6EINqIee1fAU68HyFZS8xXQSe1Ps4iTFS4Ud32JwRelY5p83QaQGeBJ0x4YXTl ug+OvJIEvcRAttOKy4OoHVKSdYexsvXfqcHBK3i0stgoZwWNCef1AJVJJFMXmvULizBp alp0qWnp0DlxOSlVZzCglahr5tgUyl580qxKOvZTbFjPgUOrx3e34+1KSVBIa/rCFKvt 48EVfMACVtlMd1AMqUOO2NY1MDfkRJomA3bTl1I4PyTSsjCX7+9PQQ3PbxZXX24TMvyf Fk7U8nix+I0vnbRSop1eWA9T55JrVQB2h4Jtke4Whl1/SPcuN4U9Nm4IbXRTyVWj6v8b QFAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=EVXXrzlGkfZYargojUNypjwD4u+3zFdZxO0ELZcNMNc=; b=MEkRo9Pt0BmrrFK+PNRSi74GGLAjl4KvHBYM6NWlJXd+zc7wmMqxVem14uZRoNXF7f PQeNrdCtZqpmYC/bhc1vFQcrJQJ7P8NGcaRryTuP9Y+8Ntk4ON/y5Dlj53zOb1GLSKYS FHsE+7Z4kV8+yQa4OpMdCBcWN0oC5UWoCQ6EfRoqITdGFRTtbEfC4fgSykP0FkserT5z Vbs3cmKj/dBxVwcGD32c4JYSkqBAJRP1O/O0/pCcpbbUFoHLGlSwHKS9YaBF7Pn65EC1 wJHRvZRjG5S4RVkKq8/dV9lESVW6gkaPIFW4hgtjJiFL/twNc2jGC+HXZ1FkbofkFRTA Zbcw== X-Gm-Message-State: AOAM530v9Y5rh3pPPUDGtoKHf4buOZAFTss0u+NwOS/AMz5ZypEFAaCL zOeGGFazbwvplWgogoC9rrhJ6w== X-Google-Smtp-Source: ABdhPJyJLf0wv4DcOr+P0n4rlRiRyrSSXMqT7lKML5dyiRvLMgz7oJ5XC6Qxsn2MmUfSZNmPBBgmfw== X-Received: by 2002:adf:f082:: with SMTP id n2mr2157873wro.326.1593153712210; Thu, 25 Jun 2020 23:41:52 -0700 (PDT) Received: from 6wind.com (2a01cb0c0005a600345636f7e65ed1a0.ipv6.abo.wanadoo.fr. [2a01:cb0c:5:a600:3456:36f7:e65e:d1a0]) by smtp.gmail.com with ESMTPSA id r3sm4305736wmh.36.2020.06.25.23.41.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jun 2020 23:41:51 -0700 (PDT) Date: Fri, 26 Jun 2020 08:41:51 +0200 From: Olivier Matz To: Vivien Didelot Cc: dev@dpdk.org, Patrick Keroulas , Ferruh Yigit Message-ID: <20200626064151.GV12564@platinum> References: <20200625190119.265739-1-vivien.didelot@gmail.com> <20200625190119.265739-2-vivien.didelot@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200625190119.265739-2-vivien.didelot@gmail.com> User-Agent: Mutt/1.10.1 (2018-07-13) Subject: Re: [dpdk-dev] [RFC PATCH 1/3] net/mlx5: add timestamp-to-ns converter from libibverbs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Vivien, On Thu, Jun 25, 2020 at 03:01:17PM -0400, Vivien Didelot wrote: > From: Patrick Keroulas > > While some devices update their own clock info to provide current time, > mlx5dv part of libibverbs already handles this and also converts any > raw counter cycle to nanoseconds. > > Signed-off-by: Patrick Keroulas > --- > drivers/common/mlx5/linux/mlx5_glue.c | 16 +++++++++++++ > drivers/common/mlx5/linux/mlx5_glue.h | 4 ++++ > drivers/net/mlx5/linux/mlx5_ethdev_os.c | 30 +++++++++++++++++++++++++ > drivers/net/mlx5/linux/mlx5_os.c | 1 + > drivers/net/mlx5/mlx5.h | 1 + > 5 files changed, 52 insertions(+) > > diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c > index c91ee33bb..cac24015b 100644 > --- a/drivers/common/mlx5/linux/mlx5_glue.c > +++ b/drivers/common/mlx5/linux/mlx5_glue.c > @@ -80,6 +80,20 @@ mlx5_glue_query_rt_values_ex(struct ibv_context *context, > return ibv_query_rt_values_ex(context, values); > } > > +static int > +mlx5_glue_get_clock_info(struct ibv_context *context, > + struct mlx5dv_clock_info *clock_info) > +{ > + return mlx5dv_get_clock_info(context, clock_info); > +} > + > +static uint64_t > +mlx5_glue_mlx5dv_ts_to_ns(struct mlx5dv_clock_info *clock_info, > + uint64_t device_timestamp) > +{ > + return mlx5dv_ts_to_ns(clock_info, device_timestamp); > +} > + > static int > mlx5_glue_query_port(struct ibv_context *context, uint8_t port_num, > struct ibv_port_attr *port_attr) > @@ -1207,6 +1221,8 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { > .query_device = mlx5_glue_query_device, > .query_device_ex = mlx5_glue_query_device_ex, > .query_rt_values_ex = mlx5_glue_query_rt_values_ex, > + .get_clock_info = mlx5_glue_get_clock_info, > + .convert_ts_to_ns = mlx5_glue_mlx5dv_ts_to_ns, > .query_port = mlx5_glue_query_port, > .create_comp_channel = mlx5_glue_create_comp_channel, > .destroy_comp_channel = mlx5_glue_destroy_comp_channel, > diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h > index 5d238a40a..8d05e7398 100644 > --- a/drivers/common/mlx5/linux/mlx5_glue.h > +++ b/drivers/common/mlx5/linux/mlx5_glue.h > @@ -123,6 +123,10 @@ struct mlx5_glue { > struct ibv_port_attr *port_attr); > struct ibv_comp_channel *(*create_comp_channel) > (struct ibv_context *context); > + int (*get_clock_info)(struct ibv_context *context, > + struct mlx5dv_clock_info *clock_info); > + uint64_t (*convert_ts_to_ns)(struct mlx5dv_clock_info *clock_info, > + uint64_t device_timestamp); > int (*destroy_comp_channel)(struct ibv_comp_channel *channel); > struct ibv_cq *(*create_cq)(struct ibv_context *context, int cqe, > void *cq_context, > diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c > index ab47cb531..86bec111b 100644 > --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c > +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c > @@ -355,6 +355,36 @@ mlx5_set_flags(struct rte_eth_dev *dev, unsigned int keep, unsigned int flags) > return mlx5_ifreq(dev, SIOCSIFFLAGS, &request); > } > > +/** > + * Convert raw clock counter to nanoseconds > + * > + * @param dev > + * Pointer to Ethernet device structure. > + * @param[in&out] timestamp > + * Pointer to the timestamp to be converted. > + * > + * @return > + * 0 if the clock has correctly been read > + * The value of errno in case of error > + */ > +int > +mlx5_convert_ts_to_ns(struct rte_eth_dev *dev, uint64_t *timestamp) > +{ > + struct mlx5_priv *priv = dev->data->dev_private; > + struct ibv_context *ctx = priv->sh->ctx; > + struct mlx5dv_clock_info clock_info; > + > + int err = mlx5_glue->get_clock_info(ctx, &clock_info); > + if (err != 0) { > + DRV_LOG(WARNING, "Could not get the clock info!"); > + return err; > + } > + > + *timestamp = mlx5_glue->convert_ts_to_ns(&clock_info, *timestamp); > + > + return err; > +} Should mlx5dv_get_clock_info() be called for each convesion? In the man page [1], we can read this: If the clock_info becomes too old then time conversion will return wrong conversion results. The user must ensure that mlx5dv_get_clock_info(3) is called at least once every max_clock_info_update_nsec as returned by the mlx5dv_query_device(3) function. I understand this as mlx5dv_get_clock_info() being slower than mlx5dv_ts_to_ns(), and should not be called for each packet. I suppose that when rdma-core library does a conversion to nanosec, it takes in account the fact that system and device are not synchronized (are they?). [1] https://man7.org/linux/man-pages/man3/mlx5dv_get_clock_info.3.html Olivier