From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EED8A04A5; Wed, 17 Jun 2020 17:58:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 032581252; Wed, 17 Jun 2020 17:58:24 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 26671100C for ; Wed, 17 Jun 2020 17:58:22 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05HFu6Kd008699; Wed, 17 Jun 2020 08:58:20 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=date : from : to : cc : subject : message-id : references : content-type : in-reply-to : mime-version; s=pfpt0818; bh=jw81fhxk30Pzk+tTBzVuyqolSfQJgonGg4R6dv7KQgk=; b=DmDyP4yGxRZcEtyZz25lIRCO1lzbKyOtdxEC8Muixiyh3a093iFwjzB4vZ1Oy/NJkEkn UmFnL/g6iuYOb7+nrmLEZsI4y9fCCij6FGNjSw56kMRx58AmWuODVVQPmruAKuUg11bL z9j162gWlLUYtx94enK/M+9zMAGm5wDKtga5AWIS5v3LfKM4tGRozc5njvaWpB8dFERx w/56/JJ27qmCwqCVL+s1yl9163FoaoX7VuuqI+wPP7AoLAhQO0MM7seawnvdHzAb6rck 6PypO2NQelaKT7rhZay9lx/65X6QrrSEoWQP1V/mGypftHtVMqMAL//yYoLcu9rnWBfh VA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 31q658n8p9-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jun 2020 08:58:20 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 17 Jun 2020 08:58:18 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 17 Jun 2020 08:58:18 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.175) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 17 Jun 2020 08:58:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LIMTKQry9RMubd5Z/W9x5PWQkZEwd/p6XH48LFqMZ00Z8V5rsPgikz7tFnSgZxTVWttyzqzhXmC2BikuzyqNk1V3/XvtPEJbahOps7vuB1jH3E/QkcSslTYnD7jYG7MrK9F++oRURqX3Mu2HZhBCLd4STSwHfLShZRghK+2FIrahy7qbw6TQSAHXosyW5d13Fh/M1hUvnIH8cbZb8jJ75u9/k99AGM7n9fKab6snvjqFkX+3bABAlI5+TSzItP4co+jTbt8fWHWOZKp+zdO9D3bVHemge72OtXSN6k4Rgz5/8WBIIznvvFNTeNmn3MggKK4+wHlkPAzOYv+d7f6Psw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jw81fhxk30Pzk+tTBzVuyqolSfQJgonGg4R6dv7KQgk=; b=Zn0KSzuNssKeJUooLLlnEAHnXuzh8NaLlFxrnPuGBzRJHGpnOigu2Os3ETj1IsQXa2nlLYBz+gd3t/vLWiXlXcty7JleGoK8Hqs3R0BgWMOOURSoCqJAX8Xp5jfyph+9OvjTmTfWuEa+XeDWSBpfokFqgracIJ0brqb1ZWAJk/H7dBwsdJa2YTlRTVBT+tBBBYM1s75fI9iYbISWLDGL2pCdcYlhXC3KnUAuVtEhh3363BJM4DQGtcmKZkBOo6dJu3siWNUBO/uDN72MnNf5zvcmpqcWLt+hFV2C+IQqJlhSuBompx9GhDQ9MTSdmNd1yP6ruuVHFRzvzgDsI80F5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jw81fhxk30Pzk+tTBzVuyqolSfQJgonGg4R6dv7KQgk=; b=loJ53q4N+MP1g9wgvEFyRbKpC6k0OBMfFqWcmR3nmIzXwbZKLx7Q31GSsLNMCs8I4lpKkMQVsv1QAuWpRDh5sSqGqb1wyJ4iVVUP28a0GCvJWt0trcU+a89f0ftpkKEH26lzrrSwMBVROddYtWz+Gh1Zm7nOTyCHbTLqJcKZK4o= Authentication-Results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=marvell.com; Received: from MWHPR18MB1070.namprd18.prod.outlook.com (2603:10b6:300:a5::11) by MWHPR18MB0941.namprd18.prod.outlook.com (2603:10b6:300:a0::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.25; Wed, 17 Jun 2020 15:58:15 +0000 Received: from MWHPR18MB1070.namprd18.prod.outlook.com ([fe80::e895:bdac:e157:2ed2]) by MWHPR18MB1070.namprd18.prod.outlook.com ([fe80::e895:bdac:e157:2ed2%11]) with mapi id 15.20.3088.029; Wed, 17 Jun 2020 15:58:15 +0000 Date: Wed, 17 Jun 2020 21:27:57 +0530 From: Harman Kalra To: Slava Ovsiienko CC: "dev@dpdk.org" , Thomas Monjalon , Matan Azrad , Raslan Darawsheh , Ori Kam , "olivier.matz@6wind.com" , Shahaf Shuler Message-ID: <20200617155756.GA190547@outlook.office365.com> References: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> <20200610133332.GA54613@outlook.office365.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.0 (2018-05-17) X-ClientProxiedBy: BM1PR01CA0084.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1::24) To MWHPR18MB1070.namprd18.prod.outlook.com (2603:10b6:300:a5::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from outlook.office365.com (115.113.156.2) by BM1PR01CA0084.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Wed, 17 Jun 2020 15:58:12 +0000 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f77da9f1-c685-4263-3d72-08d812d73e71 X-MS-TrafficTypeDiagnostic: MWHPR18MB0941: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-Forefront-PRVS: 04371797A5 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZZ7ZrHu/SuJmLHIv1hMJtAYbeOzoelAJNQy0jmlsospxSk2QFjl/CAEmFKn8rsbTPNgpvKv2a+V6woDzUk/BYuENUkniVSFIP2q5rMQRrDqPgMnopT9q+cwoFatlXpw3xQxkKa3a/FUUXTu9ZscVXgn+ryEf+xr31Sa6AyxVr7OL8QcaoKHMT0oMNYzkCelVU3W1RarF0sjki6CiyeonoUt1FCanuVv5uZcbV4t0+utZY40ifA3zRBMxOZW+a/c18spVSJuVrwra0YIq6PxvX5pQnOWvQQVpqMxkKHiu1++d7/hDjKx5i6X0i5C/s6S1UiXdXcgIns2VSh7Yqi9w2KhRuKK4URCcZvHYIXHF6c+hDfsjhXcT5RhcgrN/FErQ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR18MB1070.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(376002)(366004)(396003)(346002)(136003)(186003)(16526019)(1076003)(8676002)(956004)(53546011)(8936002)(26005)(6506007)(6916009)(83380400001)(6666004)(55236004)(5660300002)(478600001)(2906002)(33656002)(55016002)(54906003)(4326008)(86362001)(7696005)(52116002)(66476007)(66946007)(316002)(9686003)(66556008)(60764002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: eXKwcyx8SRWUkiIEaIBHsH7pSQuDWBOLMLXqTbNWHXAUVyD4IHzLwD4cp1C93EYBiXTEEZ5qGLPO+168bRMGbLKOeu1WSF8xigMdBH9rJ7E4+M/h5Qf128MKlzeIrNe94ezMzX2KN/A02gSqqn7vXLBgIhBxOSofGKbd8e1krjSBUFFdBzSTH5yR/MBl9LHjiCE1zyUFVcEuPxOP0PTWPaGrYebNWCLjv67saQCmu1+kEl4w+brSqx1otcYchM7LJrFNQPrKpIErvF0UlPHJlU8rtM+R2dqDGOmtEcVEYy7AaRZpbXMg5rrKGePcnEEUbMa106HVuzfu2FDwhox/rXaIxLG+r6ME/5pY41jPOAKN6IDFCC8Uy3GcV6swjw77dlPTenYEIooSt5N56BGSCG63wNPlNgzxgIpjPrAhc61N76QNgr/wbz2SQq7BeqM7AM+L+qq6QNgZFD6I23rhxLzGfSOLWoQQ0U77GiN3Z34= X-MS-Exchange-CrossTenant-Network-Message-Id: f77da9f1-c685-4263-3d72-08d812d73e71 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2020 15:58:15.0101 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8qX/lBZMsVXF0k1sgbKYsSKnu0vIYFKmhnG0xaZfV0y7vWHKnAIoDWdmh7SkMtXU9rQO/UP7J0MAHhWHokg4PQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB0941 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-17_06:2020-06-17, 2020-06-17 signatures=0 Subject: Re: [dpdk-dev] [EXT] RE: [RFC] mbuf: accurate packet Tx scheduling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 10, 2020 at 03:16:12PM +0000, Slava Ovsiienko wrote: > External Email > > ---------------------------------------------------------------------- > Hi, Harman > > > -----Original Message----- > > From: Harman Kalra > > Sent: Wednesday, June 10, 2020 16:34 > > To: Slava Ovsiienko > > Cc: dev@dpdk.org; Thomas Monjalon ; Matan > > Azrad ; Raslan Darawsheh > > ; Ori Kam ; > > olivier.matz@6wind.com; Shahaf Shuler > > Subject: Re: [dpdk-dev] [RFC] mbuf: accurate packet Tx scheduling > > > > On Wed, Jun 10, 2020 at 06:38:05AM +0000, Viacheslav Ovsiienko wrote: > > > > Hi Viacheslav, > > > > I have some queries below: > > > > > There is the requirement on some networks for precise traffic timing > > > management. The ability to send (and, generally speaking, receive) the > > > packets at the very precisely specified moment of time provides the > > > opportunity to support the connections with Time Division Multiplexing > > > using the contemporary general purpose NIC without involving an > > > auxiliary hardware. For example, the supporting of O-RAN Fronthaul > > > interface is one of the promising features for potentially usage of > > > the precise time management for the egress packets. > > > > > > The main objective of this RFC is to specify the way how applications > > > can provide the moment of time at what the packet transmission must be > > > started and to describe in preliminary the supporting this feature > > > from > > > mlx5 PMD side. > > > > > > The new dynamic timestamp field is proposed, it provides some timing > > > information, the units and time references (initial phase) are not > > > explicitly defined but are maintained always the same for a given port. > > > Some devices allow to query rte_eth_read_clock() that will return the > > > current device timestamp. The dynamic timestamp flag tells whether the > > > field contains actual timestamp value. For the packets being sent this > > > value can be used by PMD to schedule packet sending. > > > > > > After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation and > > > obsoleting, these dynamic flag and field will be used to manage the > > > timestamps on receiving datapath as well. > > > > > > When PMD sees the "rte_dynfield_timestamp" set on the packet being > > > sent it tries to synchronize the time of packet appearing on the wire > > > with the specified packet timestamp. It the specified one is in the > > > past it should be ignored, if one is in the distant future it should > > > be capped with some reasonable value (in range of seconds). These > > > specific cases ("too late" and "distant future") can be optionally > > > reported via device xstats to assist applications to detect the > > > time-related problems. > > > > > > There is no any packet reordering according timestamps is supposed, > > > neither within packet burst, nor between packets, it is an entirely > > > application responsibility to generate packets and its timestamps in > > > desired order. The timestamps can be put only in the first packet in > > > the burst providing the entire burst scheduling. > > > > Since its applicaiton responsibility to care of packet reordering and many > > other parameters, so why cant application itself take the responsibility of > > packet scheduling, i.e. applicaton can hold for the required time before > > calling tx-burst? Why are we even offloading this job to PMD? > > > - The scheduling is required to be very precise. Within handred(s) of nanoseconds. > - It saves CPU cycles. Application just should prepare the packets, put the desired timestamps > and call tx_burst(). "Shut-n-forget" approach. > > SW approach is potentially possible, application can hold the time and schedule packets itself. > But... Can we guarantee the stable delay between tx_burst call and data on the wire? > Should we waste CPU cycles to wait the desired moment of time? Can we guarantee > stable interrupt latency if we choose to schedule on interrupts approach? > > This RFC splits the responsibility - application should prepare the data and specify > when it desires to send, the rest is on PMD. I agree with the fact that we cannot guarantee the delay between tx burst call and data on wire, hence PMD should take care of it. Even if PMD is holding, it is wastage of CPU cycles or if we setup an alarm then also interrupt latency might be a concern to achieve precise timming. So how are you planning to address both of above issue in PMD. > > > > > > > PMD reports the ability to synchronize packet sending on timestamp > > > with new offload flag: > > > > > > This is palliative and is going to be replaced with new eth_dev API > > > about reporting/managing the supported dynamic flags and its related > > > features. This API would break ABI compatibility and can't be > > > introduced at the moment, so is postponed to 20.11. > > > > > > For testing purposes it is proposed to update testpmd "txonly" > > > forwarding mode routine. With this update testpmd application > > > generates the packets and sets the dynamic timestamps according to > > > specified time pattern if it sees the "rte_dynfield_timestamp" is registered. > > > > So what I am understanding here is "rte_dynfield_timestamp" will provide > > information about three parameters: > > - timestamp at which TX should start > > - intra packet gap > > - intra burst gap. > > > > If its about "intra packet gap" then PMD can take care, but if it is about intra > > burst gap, application can take care of it. > > Not sure - the intra-burst gap might be pretty small. > It is supposed to handle intra-burst in the same way - by specifying > the timestamps. Waiting is supposed to be implemented on tx_burst() retry. > Prepare the packets with timestamps, tx_burst - if not all packets are sent - > it means queue is waiting for the schedult, retry with the remaining packets. > As option - we can implement intra-burst wait basing rte_eth_read_clock(). Yeah, I think app can make use of rte_eth_read_clock() to implement intra-burst gap. But my actual doubt was, what all information will app provide as part of "rte_dynfield_timestamp" - one I understand will be timestamp at which packets should be sent out. What else? intra-packet gap ? Thanks Harman > > > > The new testpmd command is proposed to configure sending pattern: > > > > > > set tx_times , > > > > > > - the delay between the packets within the burst > > > specified in the device clock units. The number > > > of packets in the burst is defined by txburst parameter > > > > > > - the delay between the bursts in the device clock units > > > > > > As the result the bursts of packet will be transmitted with specific > > > delays between the packets within the burst and specific delay between > > > the bursts. The rte_eth_get_clock is supposed to be engaged to get the > > > > I think here you mean "rte_eth_read_clock". > Yes, exactly. Thank you for the correction. > > With best regards, Slava > > > > > > > Thanks > > Harman > > > > > current device clock value and provide the reference for the timestamps. > > > > > > Signed-off-by: Viacheslav Ovsiienko > > > --- > > > lib/librte_ethdev/rte_ethdev.h | 4 ++++ > > > lib/librte_mbuf/rte_mbuf_dyn.h | 16 ++++++++++++++++ > > > 2 files changed, 20 insertions(+) > > > > > > diff --git a/lib/librte_ethdev/rte_ethdev.h > > > b/lib/librte_ethdev/rte_ethdev.h index a49242b..6f6454c 100644 > > > --- a/lib/librte_ethdev/rte_ethdev.h > > > +++ b/lib/librte_ethdev/rte_ethdev.h > > > @@ -1178,6 +1178,10 @@ struct rte_eth_conf { > > > /** Device supports outer UDP checksum */ #define > > > DEV_TX_OFFLOAD_OUTER_UDP_CKSUM 0x00100000 > > > > > > +/** Device supports send on timestamp */ #define > > > +DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP 0x00200000 > > > + > > > + > > > #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001 > > /**< > > > Device supports Rx queue setup after device started*/ #define > > > RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002 diff --git > > > a/lib/librte_mbuf/rte_mbuf_dyn.h b/lib/librte_mbuf/rte_mbuf_dyn.h > > > index 96c3631..fb5477c 100644 > > > --- a/lib/librte_mbuf/rte_mbuf_dyn.h > > > +++ b/lib/librte_mbuf/rte_mbuf_dyn.h > > > @@ -250,4 +250,20 @@ int rte_mbuf_dynflag_lookup(const char *name, > > > #define RTE_MBUF_DYNFIELD_METADATA_NAME > > "rte_flow_dynfield_metadata" > > > #define RTE_MBUF_DYNFLAG_METADATA_NAME > > "rte_flow_dynflag_metadata" > > > > > > +/* > > > + * The timestamp dynamic field provides some timing information, the > > > + * units and time references (initial phase) are not explicitly > > > +defined > > > + * but are maintained always the same for a given port. Some devices > > > +allow > > > + * to query rte_eth_read_clock() that will return the current device > > > + * timestamp. The dynamic timestamp flag tells whether the field > > > +contains > > > + * actual timestamp value. For the packets being sent this value can > > > +be > > > + * used by PMD to schedule packet sending. > > > + * > > > + * After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation > > > + * and obsoleting, these dynamic flag and field will be used to > > > +manage > > > + * the timestamps on receiving datapath as well. > > > + */ > > > +#define RTE_MBUF_DYNFIELD_TIMESTAMP_NAME > > "rte_dynfield_timestamp" > > > +#define RTE_MBUF_DYNFLAG_TIMESTAMP_NAME > > "rte_dynflag_timestamp" > > > + > > > #endif > > > -- > > > 1.8.3.1 > > >