From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8D3D43758; Fri, 22 Dec 2023 04:03:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ABF6E4025D; Fri, 22 Dec 2023 04:03:49 +0100 (CET) Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by mails.dpdk.org (Postfix) with ESMTP id 0970B4003C for ; Fri, 22 Dec 2023 04:03:48 +0100 (CET) Received: by mail-qk1-f170.google.com with SMTP id af79cd13be357-781048954d9so94920885a.1 for ; Thu, 21 Dec 2023 19:03:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uconn-edu.20230601.gappssmtp.com; s=20230601; t=1703214228; x=1703819028; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CkJTbI9SKZyzL3haxo/bev+yPojbn7aLU1Hqj+QP9uY=; b=NglrXASsO3rDGQNXnh9b9xcmYg134xfnM19dI4oGLi1DL1e6+hHseIhIhouCPIVGYq lxMn8kCpmsbgELT+0nyVpPM5QXt2VUGEGS0zn9VMzdMF8K2ZyqJ9ggBbylOW62zc3Y1f KwFyizvuCmmquYlx9Jxs0KhaR9X5YaZJkgfAxI8+naG4C/tKlD20BKh6YFezTk0JG+33 sLMZfqQuCEgB+GbEuBFodJdSu246OsykWjSzhv2yGXCXXkKIc45AZtCsfLPmPy/kTBYr BfJoKlt51UCfLdXd0B7BLlRRHJwWWXwN1jU2UX2v9DZx8QLV6VE9E4s2TnvJG85Ac5iy TXsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703214228; x=1703819028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CkJTbI9SKZyzL3haxo/bev+yPojbn7aLU1Hqj+QP9uY=; b=MRN/xQjDbXiathyyk0tKAP0LCajUbwxOVxUvmypbgloswPOnv5L+QKAQaSrnUXM75k A+lCftnM+RTsXMKhb3/wfIVHD4tvMb+rrbUau87cKm4UvXi1LhL6LQoguiUngGVQpy3m +gL1XoQWN+G4OVno+VB1ZtMkQ9fnsw+MtESZPoMckWlFpWISYZz1QtGpDoqYGx0j5k7j XByFZLtrQGUyXcAMWTioOAFVv16c61mQf3WYRmk+JsJn3Vkpk/A5GWn6aCzA//Z2xfZq HEoeykrn1plQGdxRhj08dX4n9SMqztwzw2bOBTQ2r2eYo8JlOf6VonfEFBA3nj78UNHQ /SXw== X-Gm-Message-State: AOJu0YztgK0EhdwBYnnr+xI947BZHH61uyWQPrhe6f6O5BccGHVRr3Kg /WgRA4y/gTeS2Mj5ioFyqft9Bfld58TLxg== X-Google-Smtp-Source: AGHT+IHHy4OnetI6aPXpItJd0B/duHHCcz6TbsbEEo3glHuqs8Nntvi4Zfu+wl74T4yjOw6etHcjSw== X-Received: by 2002:a05:620a:400a:b0:781:23b0:c883 with SMTP id h10-20020a05620a400a00b0078123b0c883mr1083587qko.74.1703214228292; Thu, 21 Dec 2023 19:03:48 -0800 (PST) Received: from localhost.localdomain ([137.99.252.108]) by smtp.gmail.com with ESMTPSA id b21-20020a05620a271500b007811eef5044sm816820qkp.72.2023.12.21.19.03.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 19:03:47 -0800 (PST) From: Chuanyu Xue To: simei.su@intel.com Cc: beilei.xing@intel.com, chuanyu.xue@uconn.edu, dev@dpdk.org, qi.z.zhang@intel.com, wenzhuo.lu@intel.com Subject: RE: [PATCH] net/e1000: support launchtime feature Date: Thu, 21 Dec 2023 22:03:46 -0500 Message-Id: <20231222030346.12805-1-chuanyu.xue@uconn.edu> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Simei, Thank you so much for your review. >> >> /* QAV Tx mode control register */ >> #define E1000_I210_TQAVCTRL 0x3570 >> +#define E1000_I210_LAUNCH_OS0 0x3578 > >What does this register mean? > "LAUNCH_OS0" is defined as LaunchOffset register, which sets the base time for launchtime. Based on i210 datasheet V3.7 Sec 7.2.2.2.3, the actual launch time is computed as 32 * (LaunchOffset + LaunchTime). In this context, the register is used to set the LaunchOffset later as 0. >> >> + if (igb_tx_timestamp_dynflag > 0) { >> + tqavctrl = E1000_READ_REG(hw, E1000_I210_TQAVCTRL); >> + tqavctrl |= E1000_TQAVCTRL_MODE; >> + tqavctrl |= E1000_TQAVCTRL_FETCH_ARB; /* Fetch the queue most >> empty, no Round Robin*/ >> + tqavctrl |= E1000_TQAVCTRL_LAUNCH_TIMER_ENABLE; /* Enable >> launch time */ > > In kernel driver, "E1000_TQAVCTRL_DATATRANTIM (BIT(9))" and > "E1000_TQAVCTRL_FETCHTIME_DELTA (0xFFFF << 16)" are set, does it have some > other intention here? "E1000_TQAVCTRL_DATATRANTIM" is same as "E1000_TQAVCTRL_LAUNCH_TIMER_ENABLE" "E1000_TQAVCTRL_FETCHTIME_DELTA" maximizes the data fetch time. If "E1000_TQAVCTRL_FETCH_ARB" is set, there is no need to set this field, because the arbitrary fetching prioritizes the most empty queue, regardless of the fetch time. (referring Sec 7.2.7.5) I have also tested aligning with the kernel driver settings using (0xFFFF << 16) and omitting 'E1000_TQAVCTRL_FETCH_ARB', the launchtime feature also worked as expected. However, the arbitrary fetch mode seems more suitable as DPDK lacks an interface to set fetch delay, unlike in the kernel which can be configured (e.g., through 'Delta' in ETF Qdisc). Any suggestions here? >> +static int >> +eth_igb_read_clock(__rte_unused struct rte_eth_dev *dev, uint64_t >> +*clock) { >> + uint64_t systime_cycles; >> + struct e1000_adapter *adapter = dev->data->dev_private; >> + >> + systime_cycles = igb_read_systime_cyclecounter(dev); >> + uint64_t ns = rte_timecounter_update(&adapter->systime_tc, >> systime_cycles); > >Do you also run "ptp timesync" when testing this launchtime feature? > I used `rte_eth_timesync_enable` function during the test. I am not familiar with the `ptp timesync` in DPDK. If you are referring to something else, could you please guide me on how to test it? >> >> +/* Macro to compensate latency in launch time offloading*/ >> +#define E1000_I210_LT_LATENCY 0x41F9 > >What does this value depend on? > Through my test, I observed a constant latency between the launchtime and the actual Tx time measured by the `rte_eth_timesync_read_tx_timestamp` function. I didn't find a description of this latency in the datasheet. In my test, the latency appears to be relative to the data rate, and independent from the packet size and throughput. The latency slightly changed in different experiments, but in each experiment, it remained constant for all the Tx packets. I also tested this latency consistently on two different NICs (I210 GE-1T-X1, I210 X1-V2). Here are some measurement results (in ns): +-----------+---------------+---------------+---------------+---------------+---------------+ | Data Rate | Measurement 1 | Measurement 2 | Measurement 3 | Measurement 4 | Measurement 5 | +-----------+---------------+---------------+---------------+---------------+---------------+ | 10M | 14400 | 14544 | 14384 | 14896 | 14336 | +-----------+---------------+---------------+---------------+---------------+---------------+ | 100M | 31016 | 31016 | 31000 | 31000 | 31048 | +-----------+---------------+---------------+---------------+---------------+---------------+ | 1G | 16880 | 16880 | 16880 | 16880 | 16880 | +-----------+---------------+---------------+---------------+---------------+---------------+ Any suggestions here? Is it supposed to be embedded directly here or left to the application level to compensate? I can fix it accordingly. - Chuanyu