Re: [PATCH 1/3] ethdev: add frequency adjustment API

[Ferruh Yigit <ferruh.yigit@amd.com>]

On 8/9/2023 6:06 AM, Simei Su wrote:
> This patch introduces a new timesync API "rte_eth_timesync_adjust_fine"
> which enables finer adjustment of the PHC clock. During PTP timesync,
> "rte_eth_timesync_adjust_time" focuses on phase adjustment while
> "rte_eth_timesync_adjust_fine" focuses on frequency adjustment.
> 

Hi Simei,

The patch adds new API and new dev_ops, mechanics of it mostly looks
good, there is minor comment below.

But there is not much comment on it because concept may be not well
known as it is not much known to me. Can you please try to add more
details and documentations to help users and driver developers?


I have some understanding based on your comments, can you please check
if it is correct:

In PTP protocol master and client synchronize clocks periodically, and
adjust the delta with 'rte_eth_timesync_adjust_time()' API, which simply
use this delta as offset in its time function.

But new API added by this patch, 'rte_eth_timesync_adjust_fine()', adds
a new synchronization method by saying device to change its HW timer
frequency.

The name of the API says this is a finer adjustment method, I wonder
why? I don't know much about timer HW but my understanding is they are
pretty accurate, why changing frequency gives a finer adjustment? Is it
common that difference frequency of HW timers cause drift by time?

Btw, what is PHC clock? Is it "Pulse Hollow Cathode" (as chatGPT
suggests :), is there any significance of this type of clock? Is this
API valid only for this type of clock?


And for users, assuming the HW supports both methods, how should they
decide which one to use? Can we provide some guidance to them? I can see
API suggest to use 'rte_eth_timesync_adjust_fine()' when time diff
offset is less than a threshold but what is the threshold and what
happens if user only uses 'rte_eth_timesync_adjust_time()'?
Is there a more real life numbers to provide as samples, maybe can
provide them in the commit log as sample if we can't provide them in API
documentation?


> This new function gets the scaled_ppm (desired frequency offset from
> nominal frequency in parts per million, but with a 16 bit binary
> fractional field).
> 

What is "scaled parts per million"? I think I understand what parameter
does, but not sure about what is the unit here?

Also PTP client sample mentions about algorithms to use for frequency
adjustment, like PI servo algorithm next patch uses, what are these
algorithms?
Should we mention from algorithms or specifically from "PI servo
algorithm" in the API documentation?

> Signed-off-by: Simei Su <simei.su@intel.com>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> ---
>  lib/ethdev/ethdev_driver.h       |  5 +++++
>  lib/ethdev/ethdev_trace.h        |  9 +++++++++
>  lib/ethdev/ethdev_trace_points.c |  3 +++
>  lib/ethdev/rte_ethdev.c          | 19 +++++++++++++++++++
>  lib/ethdev/rte_ethdev.h          | 38 ++++++++++++++++++++++++++++++++++++++

'version.map' also should be updated to export the API, so that DPDK
applications linked with shared DPDK library can use it.

CI already complaining about this:
https://mails.dpdk.org/archives/test-report/2023-August/435936.html


>  5 files changed, 74 insertions(+)
> 
> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> index 980f837..4c39c3d 100644
> --- a/lib/ethdev/ethdev_driver.h
> +++ b/lib/ethdev/ethdev_driver.h
> @@ -636,6 +636,9 @@ typedef int (*eth_timesync_read_tx_timestamp_t)(struct rte_eth_dev *dev,
>  /** @internal Function used to adjust the device clock. */
>  typedef int (*eth_timesync_adjust_time)(struct rte_eth_dev *dev, int64_t);
>  
> +/** @internal Function used to adjust the clock increment rate. */
> +typedef int (*eth_timesync_adjust_fine)(struct rte_eth_dev *dev, int64_t);
> +
>  /** @internal Function used to get time from the device clock. */
>  typedef int (*eth_timesync_read_time)(struct rte_eth_dev *dev,
>  				      struct timespec *timestamp);
> @@ -1347,6 +1350,8 @@ struct eth_dev_ops {
>  	eth_timesync_read_tx_timestamp_t timesync_read_tx_timestamp;
>  	/** Adjust the device clock */
>  	eth_timesync_adjust_time   timesync_adjust_time;
> +	/** Adjust the clock increment rate */
> +	eth_timesync_adjust_fine   timesync_adjust_fine;
>  	/** Get the device clock time */
>  	eth_timesync_read_time     timesync_read_time;
>  	/** Set the device clock time */
> diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h
> index 423e712..d613eb2 100644
> --- a/lib/ethdev/ethdev_trace.h
> +++ b/lib/ethdev/ethdev_trace.h
> @@ -2175,6 +2175,15 @@ RTE_TRACE_POINT_FP(
>  	rte_trace_point_emit_int(ret);
>  )
>  
> +/* Called in loop in examples/ptpclient */
> +RTE_TRACE_POINT_FP(
> +	rte_eth_trace_timesync_adjust_fine,
> +	RTE_TRACE_POINT_ARGS(uint16_t port_id, int64_t scaled_ppm, int ret),
> +	rte_trace_point_emit_u16(port_id);
> +	rte_trace_point_emit_i64(scaled_ppm);
> +	rte_trace_point_emit_int(ret);
> +)
> +
>  /* Called in loop in app/test-flow-perf */
>  RTE_TRACE_POINT_FP(
>  	rte_flow_trace_create,
> diff --git a/lib/ethdev/ethdev_trace_points.c b/lib/ethdev/ethdev_trace_points.c
> index 91f71d8..eb539bb 100644
> --- a/lib/ethdev/ethdev_trace_points.c
> +++ b/lib/ethdev/ethdev_trace_points.c
> @@ -406,6 +406,9 @@ RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_read_tx_timestamp,
>  RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_adjust_time,
>  	lib.ethdev.timesync_adjust_time)
>  
> +RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_adjust_fine,
> +	lib.ethdev.timesync_adjust_fine)
> +
>  RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_read_time,
>  	lib.ethdev.timesync_read_time)
>  
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index 0840d2b..7b75214 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -6109,6 +6109,25 @@ rte_eth_timesync_adjust_time(uint16_t port_id, int64_t delta)
>  }
>  
>  int
> +rte_eth_timesync_adjust_fine(uint16_t port_id, int64_t scaled_ppm)
> +{
> +	struct rte_eth_dev *dev;
> +	int ret;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> +	dev = &rte_eth_devices[port_id];
> +
> +	if (*dev->dev_ops->timesync_adjust_fine == NULL)
> +		return -ENOTSUP;
> +	ret = eth_err(port_id, (*dev->dev_ops->timesync_adjust_fine)(dev,
> +								scaled_ppm));
> +
> +	rte_eth_trace_timesync_adjust_fine(port_id, scaled_ppm, ret);
> +
> +	return ret;
> +}
> +
> +int
>  rte_eth_timesync_read_time(uint16_t port_id, struct timespec *timestamp)
>  {
>  	struct rte_eth_dev *dev;
> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> index 3d44979..78fc07c 100644
> --- a/lib/ethdev/rte_ethdev.h
> +++ b/lib/ethdev/rte_ethdev.h
> @@ -5145,6 +5145,44 @@ int rte_eth_timesync_read_tx_timestamp(uint16_t port_id,
>  int rte_eth_timesync_adjust_time(uint16_t port_id, int64_t delta);
>  
>  /**
> + * Adjust the frequency of the PHC cycle counter by the indicated amount
> + * from the base frequency.
> + *
> + * This function is used to do hardware timestamp adjustment in fine
> + * granularity. It can be used in conjunction with rte_eth_timesync_adjust_time
> + * to do more precise time control.
> + *
> + * E.g, below is a simple usage:
> + * if master offset > master offset threshold
> + *	do rte_eth_timesync_adjust_time;
> + * else
> + *	do rte_eth_timesync_adjust_fine;
> + *
> + * The user can apply a control algorithm to leverage these two APIs, one
> + * example is in dpdk-ptpclient.
> + *
> + * This API is implemented with the below basic logic:
> + *   - Determine a base frequency value
> + *   - Multiply this by the abs() of the requested adjustment, then divide by
> + *     the appropriate divisor (65536 billion).
> + *   - Add or subtract this difference from the base frequency to calculate a
> + *     new adjustment.
> + *
> + * @param port_id
> + *  The port identifier of the Ethernet device.
> + * @param scaled_ppm
> + *  Desired frequency change in scaled parts per million. Scaled parts per
> + *  million is ppm with a 16-bit binary fractional field.
> + *
> + * @return
> + *   - 0: Success.
> + *   - -ENODEV: The port ID is invalid.
> + *   - -EIO: if device is removed.
> + *   - -ENOTSUP: The function is not supported by the Ethernet driver.
> + */
> +int rte_eth_timesync_adjust_fine(uint16_t port_id, int64_t scaled_ppm);
> +
> +/**
>   * Read the time from the timesync clock on an Ethernet device.
>   *
>   * This is usually used in conjunction with other Ethdev timesync functions to