From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DADEAA00BE; Tue, 7 Jul 2020 14:46:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3452E1DCB9; Tue, 7 Jul 2020 14:46:47 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70041.outbound.protection.outlook.com [40.107.7.41]) by dpdk.org (Postfix) with ESMTP id 5A5731DC43 for ; Tue, 7 Jul 2020 14:46:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wc/cojgoU6NnnoYyqnEa2xhuzOHW895CLVUd+M6wvItQb17rMNb2jfqt7qyRRq+LbGhA957RY7Dx87IzLjFs+fqwDptrduyq81fByh+dfitd3Q49uxNF6HqkLch33x9CgOMjdPBZgih/EVfLV9tNBMjroppVj8qBIJMw20ymqhzgrjYAd3Jm/sgqlMUpaDiOI2iJCSJXc9LSq0jVHYclH9aH3gJP+CnCoeFAsiyrt8n8nfyruNcsdRt3vDbtADj/rWJwi5sZY4KdWiHuK4pmQU+evljFhU8wTz+PEd1b8eniRslyT/mo0iMCHjuADFj4mblGPUZUv5CNrN6fETeZTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qDIW7/aLhi96gjPa3FbqeA/qkmQjmpWtXcV/Byq1yls=; b=h3uTc9d0UVIptjNzfLQlPix9ISiXw7aMzCyn7QWvimxtUOr0nnJayKwIlciyGNAhRj9mkUxRz5UEqDs85IofVHD3EW36R2Pntp9RFufUoYAr9GTpdkFHGamaW94mzDxQolPpVAXqo0yWfM9Qg0xE58NE8Ms+W7ZGSo+JbuzwC35GZI9v5iKGZoELSSCPiXglmKM1R+CtjYem2kkcTuUiXZZZFMpwmiNTF7CqWFXrD6Ve2xDOQEO7RJ305F0CyhMp1Q3QCz+ENJFYY57EwpgWQ1ey16aQ6CX5ORuWi3B+rwF9Ott3DBUQwNNED/nZ6t+CCaUwc/KQwfvUhWLtHTnIsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qDIW7/aLhi96gjPa3FbqeA/qkmQjmpWtXcV/Byq1yls=; b=ZL4gtpW9zhfMn10MOAoHNl9HnJ9Q6yB0OtFip3EItkNoMDvdKfYHx8WQ90ZJ/udNhV04BDHjI7NvHMc5mRxjAp9zibQRGljduxZOlCw27UU+zTjDkpzOqeJhGdWcmpRmg1HBfCMQMAg7em0zSk0NSgbGaeTf0a9w6afnRSdhIko= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (2603:10a6:205:8::26) by AM0PR05MB4594.eurprd05.prod.outlook.com (2603:10a6:208:b0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3153.24; Tue, 7 Jul 2020 12:46:43 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::194e:dc46:7543:50ed]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::194e:dc46:7543:50ed%2]) with mapi id 15.20.3153.029; Tue, 7 Jul 2020 12:46:43 +0000 From: Slava Ovsiienko To: Olivier Matz CC: "dev@dpdk.org" , Matan Azrad , Raslan Darawsheh , "bernard.iremonger@intel.com" , "thomas@mellanox.net" Thread-Topic: [PATCH 1/2] mbuf: introduce accurate packet Tx scheduling Thread-Index: AQHWVFTfqhZdQiUPWE6eLIBYMQWY8Kj8Akiw Date: Tue, 7 Jul 2020 12:46:43 +0000 Message-ID: References: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> <1593617787-3252-1-git-send-email-viacheslavo@mellanox.com> <20200707115046.GI5869@platinum> In-Reply-To: <20200707115046.GI5869@platinum> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: 6wind.com; dkim=none (message not signed) header.d=none;6wind.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d2b71e36-8feb-4349-0e72-08d82273cd5e x-ms-traffictypediagnostic: AM0PR05MB4594: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0457F11EAF x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: gV5bkPJTL5N300nJVwy3HcIvzwqIlhacPiMMhDOqlGscQdDhpXoezhU9Gzy9hL1Ied8k05GLDmn+Ag2D8Uy7TF8TE5355qFfhEh51hZr9IDhreF9GQDb+v5wu5rlNpeIiqjKH0BA+w+vMCiHZM5tBtKw28ABX4UfwlxUbd2Gye1/0fqzmZ94gSUByPZPeGib7hovrW8elAd4kxrO3LjgaWO6shYGv7UenXDXjTS6zFnji5L9Agukvlp1vqanvZX+TBi0F+3hM9lgadClkFCHP45fHienCXqjFGaZO3t1sdf72JlJTS6NwsMV7ET5aAHNRjcJs87z+AzupXuKuDgUAA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR05MB3265.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(498600001)(9686003)(54906003)(86362001)(8676002)(33656002)(53546011)(83380400001)(6506007)(26005)(52536014)(6916009)(5660300002)(66476007)(2906002)(66446008)(66556008)(55016002)(64756008)(66946007)(76116006)(71200400001)(8936002)(7696005)(4326008)(186003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: f0PSOKmD13Q9UpNjv8+zdPqnOZfX0E4bSGW6kLEW81hVy6cn9wGzDpyoBTieSXcVIWYHhgFBa6k84iCddx/NtEf3U+6CZGmby7JyR23AZCp8PGW0tbzhIKGOdaVePg7D1QtuUtTn5KjNHOFnUjSORPMosxMUADPvOPqYXcPrKsxpH+S/Lc/e8Q9f9OworNieJzFaoYurpqBrAJpWZZP3rvavdQATWxoU9XetG5zusVrmgeRPJ2oMfedbDUdvWBCorLE0Vj3JsU8qo/b1ojLcRCjPPpKvVw9S9JIsfCmKh+oBxYFQQFZnRXWzybkTjamCaWKKXvuSDThHRb68eleJFFI6La+TcDmUr6PdPHevTGTqhdWSVi656PN+B4uo85LH2tHE7ZUqrdgzFJyPjUOcizjvgEze05IaytqViKc5841PE0wGKkQNUMoWaQZTSDNB35CGMfpMEX5KDvsspzKR5NAviPmf1D3igfdORQOjItQ= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM4PR05MB3265.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d2b71e36-8feb-4349-0e72-08d82273cd5e X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jul 2020 12:46:43.4042 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OMNF5Co+b6E42hBjUkYn5FHmqcKOi/blN7opEHOPUyZv2CoEGpDWPxRj89qAAvY7tK+df3lhtb82ecun6Wq58MCBbmkkjwH8JVQDWLuatqA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4594 Subject: Re: [dpdk-dev] [PATCH 1/2] mbuf: introduce accurate packet Tx scheduling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, Olivier Thanks a lot for the review. > -----Original Message----- > From: Olivier Matz > Sent: Tuesday, July 7, 2020 14:51 > To: Slava Ovsiienko > Cc: dev@dpdk.org; Matan Azrad ; Raslan > Darawsheh ; bernard.iremonger@intel.com; > thomas@mellanox.net > Subject: Re: [PATCH 1/2] mbuf: introduce accurate packet Tx scheduling >=20 > Hi Slava, >=20 > Few question/comments below. >=20 > On Wed, Jul 01, 2020 at 03:36:26PM +0000, Viacheslav Ovsiienko wrote: > > There is the requirement on some networks for precise traffic timing > > management. The ability to send (and, generally speaking, receive) the > > packets at the very precisely specified moment of time provides the > > opportunity to support the connections with Time Division Multiplexing > > using the contemporary general purpose NIC without involving an > > auxiliary hardware. For example, the supporting of O-RAN Fronthaul > > interface is one of the promising features for potentially usage of > > the precise time management for the egress packets. > > > > The main objective of this RFC is to specify the way how applications > > can provide the moment of time at what the packet transmission must be > > started and to describe in preliminary the supporting this feature > > from > > mlx5 PMD side. > > > > The new dynamic timestamp field is proposed, it provides some timing > > information, the units and time references (initial phase) are not > > explicitly defined but are maintained always the same for a given port. > > Some devices allow to query rte_eth_read_clock() that will return the > > current device timestamp. The dynamic timestamp flag tells whether the > > field contains actual timestamp value. For the packets being sent this > > value can be used by PMD to schedule packet sending. > > > > After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation and > > obsoleting, these dynamic flag and field will be used to manage the > > timestamps on receiving datapath as well. >=20 > Do you mean the same flag will be used for both Rx and Tx? I wonder if i= t's a > good idea: if you enable the timestamp on Rx, the packets will be flagged > and it will impact Tx, except if the application explicitly resets the fl= ag in all > mbufs. Wouldn't it be safer to have an Rx flag and a Tx flag? A little bit difficult to say ambiguously, I thought about and did not make= strong decision. We have the flag sharing for the Rx/Tx metadata and just follow the same ap= proach. OK, I will promote ones to: RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME RTE_MBUF_DYNFIELD_TX_TIMESTAMP_NAME And, possible, we should reconsider metadata dynamic flags. >=20 > > When PMD sees the "rte_dynfield_timestamp" set on the packet being > > sent it tries to synchronize the time of packet appearing on the wire > > with the specified packet timestamp. It the specified one is in the > > past it should be ignored, if one is in the distant future it should > > be capped with some reasonable value (in range of seconds). These > > specific cases ("too late" and "distant future") can be optionally > > reported via device xstats to assist applications to detect the > > time-related problems. >=20 > I think what to do with packets to be send in the "past" could be configu= rable > through an ethdev API in the future (drop or send). Yes, currently there is no complete understanding how to handle packets out= of the time slot. In 20.11 we are going to introduce the time-based rte flow API to manage ou= t-of-window packets. =20 >=20 > > There is no any packet reordering according timestamps is supposed, > > neither within packet burst, nor between packets, it is an entirely > > application responsibility to generate packets and its timestamps in > > desired order. The timestamps can be put only in the first packet in > > the burst providing the entire burst scheduling. "can" should be replaced with "might". Current mlx5 implementation checks each packet in the burst for the timestamp presence. >=20 > This constraint makes sense. At first glance, it looks it is imposed by a= PMD or > hw limitation, but thinking more about it, I think it is the correct beha= vior to > have. Packets are ordered within a PMD queue, and the ability to set the > timestamp for one packet to delay the subsequent ones looks useful. >=20 > Should this behavior be documented somewhere? Maybe in the API > comment documenting the dynamic flag? It is documented in mlx5.rst (coming soon), do you think it should be more common? OK, will update. >=20 > > PMD reports the ability to synchronize packet sending on timestamp > > with new offload flag: > > > > This is palliative and is going to be replaced with new eth_dev API > > about reporting/managing the supported dynamic flags and its related > > features. This API would break ABI compatibility and can't be > > introduced at the moment, so is postponed to 20.11. > > > > For testing purposes it is proposed to update testpmd "txonly" > > forwarding mode routine. With this update testpmd application > > generates the packets and sets the dynamic timestamps according to > > specified time pattern if it sees the "rte_dynfield_timestamp" is regis= tered. > > > > The new testpmd command is proposed to configure sending pattern: > > > > set tx_times , > > > > - the delay between the packets within the burst > > specified in the device clock units. The number > > of packets in the burst is defined by txburst parameter > > > > - the delay between the bursts in the device clock units > > > > As the result the bursts of packet will be transmitted with specific > > delays between the packets within the burst and specific delay between > > the bursts. The rte_eth_get_clock is supposed to be engaged to get the > > current device clock value and provide the reference for the timestamps= . > > > > Signed-off-by: Viacheslav Ovsiienko > > --- > > lib/librte_ethdev/rte_ethdev.c | 1 + lib/librte_ethdev/rte_ethdev.h > > | 4 ++++ lib/librte_mbuf/rte_mbuf_dyn.h | 16 ++++++++++++++++ > > 3 files changed, 21 insertions(+) > > > > diff --git a/lib/librte_ethdev/rte_ethdev.c > > b/lib/librte_ethdev/rte_ethdev.c index 8e10a6f..02157d5 100644 > > --- a/lib/librte_ethdev/rte_ethdev.c > > +++ b/lib/librte_ethdev/rte_ethdev.c > > @@ -162,6 +162,7 @@ struct rte_eth_xstats_name_off { > > RTE_TX_OFFLOAD_BIT2STR(UDP_TNL_TSO), > > RTE_TX_OFFLOAD_BIT2STR(IP_TNL_TSO), > > RTE_TX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM), > > + RTE_TX_OFFLOAD_BIT2STR(SEND_ON_TIMESTAMP), > > }; > > > > #undef RTE_TX_OFFLOAD_BIT2STR > > diff --git a/lib/librte_ethdev/rte_ethdev.h > > b/lib/librte_ethdev/rte_ethdev.h index a49242b..6f6454c 100644 > > --- a/lib/librte_ethdev/rte_ethdev.h > > +++ b/lib/librte_ethdev/rte_ethdev.h > > @@ -1178,6 +1178,10 @@ struct rte_eth_conf { > > /** Device supports outer UDP checksum */ #define > > DEV_TX_OFFLOAD_OUTER_UDP_CKSUM 0x00100000 > > > > +/** Device supports send on timestamp */ #define > > +DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP 0x00200000 > > + > > + > > #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001 > /**< > > Device supports Rx queue setup after device started*/ #define > > RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002 diff --git > > a/lib/librte_mbuf/rte_mbuf_dyn.h b/lib/librte_mbuf/rte_mbuf_dyn.h > > index 96c3631..fb5477c 100644 > > --- a/lib/librte_mbuf/rte_mbuf_dyn.h > > +++ b/lib/librte_mbuf/rte_mbuf_dyn.h > > @@ -250,4 +250,20 @@ int rte_mbuf_dynflag_lookup(const char *name, > > #define RTE_MBUF_DYNFIELD_METADATA_NAME > "rte_flow_dynfield_metadata" > > #define RTE_MBUF_DYNFLAG_METADATA_NAME > "rte_flow_dynflag_metadata" > > > > +/* > > + * The timestamp dynamic field provides some timing information, the > > + * units and time references (initial phase) are not explicitly > > +defined > > + * but are maintained always the same for a given port. Some devices > > +allow > > + * to query rte_eth_read_clock() that will return the current device > > + * timestamp. The dynamic timestamp flag tells whether the field > > +contains > > + * actual timestamp value. For the packets being sent this value can > > +be > > + * used by PMD to schedule packet sending. > > + * > > + * After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation > > + * and obsoleting, these dynamic flag and field will be used to > > +manage > > + * the timestamps on receiving datapath as well. > > + */ > > +#define RTE_MBUF_DYNFIELD_TIMESTAMP_NAME > "rte_dynfield_timestamp" > > +#define RTE_MBUF_DYNFLAG_TIMESTAMP_NAME > "rte_dynflag_timestamp" > > + >=20 > I realize that's not the case for rte_flow_dynfield_metadata, but I think= it > would be good to have a doxygen-like comment. OK, will extend comment with expected PMD behavior and replace "/*" with "= /**" >=20 >=20 >=20 > Regards, > Olivier With best regards, Slava