From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9926EA0350; Wed, 24 Jun 2020 20:32:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF05C1D9C2; Wed, 24 Jun 2020 20:32:30 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 9FDA61D9BD for ; Wed, 24 Jun 2020 20:32:28 +0200 (CEST) IronPort-SDR: JBJPvUJzBjAhfIs+x91okkymu/EaR3GP1t4laI3H8GT+l5ebIA4BxXD6P71yyMyAVOXh5ulcKA R5G7Eu75hDZQ== X-IronPort-AV: E=McAfee;i="6000,8403,9662"; a="142807007" X-IronPort-AV: E=Sophos;i="5.75,276,1589266800"; d="scan'208";a="142807007" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jun 2020 11:32:27 -0700 IronPort-SDR: 7EiWCz8AMPOEL0rwcVPVZpLVFIKBRRviNQfPL8slkp89PMrWnqKjs0PGClZzbHMmPM7DRT7f+K 2ZU6VGow9wjg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,276,1589266800"; d="scan'208";a="354218028" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.213.244.180]) ([10.213.244.180]) by orsmga001.jf.intel.com with ESMTP; 24 Jun 2020 11:32:24 -0700 From: Ferruh Yigit To: Chengchang Tang , Andrew Rybchenko , dev@dpdk.org Cc: linuxarm@huawei.com, thomas@monjalon.net, Olivier MATZ , Matan Azrad , Jerin Jacob , Shy Shyman , Qi Zhang , Konstantin Ananyev References: <1592894934-57856-1-git-send-email-tangchengchang@huawei.com> <566d8162-7550-6812-bf62-e0fb55e8e500@solarflare.com> <4650f0a7-64cf-831d-631c-243ea510fd38@intel.com> Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJsBBMBCgBWAhsDAh4BAheABQsJCAcDBRUK CQgLBRYCAwEABQkKqZZ8FiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl6ha3sXGHZrczovL2tl eXMub3BlbnBncC5vcmcACgkQ+TPrQ98TYR8uLA//QwltuFliUWe60xwmu9sY38c1DXvX67wk UryQ1WijVdIoj4H8cf/s2KtyIBjc89R254KMEfJDao/LrXqJ69KyGKXFhFPlF3VmFLsN4XiT PSfxkx8s6kHVaB3O183p4xAqnnl/ql8nJ5ph9HuwdL8CyO5/7dC/MjZ/mc4NGq5O9zk3YRGO lvdZAp5HW9VKW4iynvy7rl3tKyEqaAE62MbGyfJDH3C/nV/4+mPc8Av5rRH2hV+DBQourwuC ci6noiDP6GCNQqTh1FHYvXaN4GPMHD9DX6LtT8Fc5mL/V9i9kEVikPohlI0WJqhE+vQHFzR2 1q5nznE+pweYsBi3LXIMYpmha9oJh03dJOdKAEhkfBr6n8BWkWQMMiwfdzg20JX0o7a/iF8H 4dshBs+dXdIKzPfJhMjHxLDFNPNH8zRQkB02JceY9ESEah3wAbzTwz+e/9qQ5OyDTQjKkVOo cxC2U7CqeNt0JZi0tmuzIWrfxjAUulVhBmnceqyMOzGpSCQIkvalb6+eXsC9V1DZ4zsHZ2Mx Hi+7pCksdraXUhKdg5bOVCt8XFmx1MX4AoV3GWy6mZ4eMMvJN2hjXcrreQgG25BdCdcxKgqp e9cMbCtF+RZax8U6LkAWueJJ1QXrav1Jk5SnG8/5xANQoBQKGz+yFiWcgEs9Tpxth15o2v59 gXK5Ag0EV9ZMvgEQAKc0Db17xNqtSwEvmfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ES YpV8QWj0xK4YM0dLxnDU2IYxjEshSB1TqAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4Ai bPtrHuIXWQOBECcVZTTOdZYGAzaYzxiAONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxD UQljeNvKYt1lZE/gAUUxNLWsYyTT+22/vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/ 3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35piVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVj sM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQI3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdc q9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYHfVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH7 1PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZqw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFB VOQOxCvwRG2QCgcJ/UTn5vlivul+cThi6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI 8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJlRr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYC GwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNhHwUCXqFrngUJCKxSYAAKCRD5M+tD3xNhH3YWD/9b cUiWaHJasX+OpiuZ1Li5GG3m9aw4lR/k2lET0UPRer2Jy1JsL+uqzdkxGvPqzFTBXgx/6Byz EMa2mt6R9BCyR286s3lxVS5Bgr5JGB3EkpPcoJT3A7QOYMV95jBiiJTy78Qdzi5LrIu4tW6H o0MWUjpjdbR01cnj6EagKrDx9kAsqQTfvz4ff5JIFyKSKEHQMaz1YGHyCWhsTwqONhs0G7V2 0taQS1bGiaWND0dIBJ/u0pU998XZhmMzn765H+/MqXsyDXwoHv1rcaX/kcZIcN3sLUVcbdxA WHXOktGTQemQfEpCNuf2jeeJlp8sHmAQmV3dLS1R49h0q7hH4qOPEIvXjQebJGs5W7s2vxbA 5u5nLujmMkkfg1XHsds0u7Zdp2n200VC4GQf8vsUp6CSMgjedHeF9zKv1W4lYXpHp576ZV7T GgsEsvveAE1xvHnpV9d7ZehPuZfYlP4qgo2iutA1c0AXZLn5LPcDBgZ+KQZTzm05RU1gkx7n gL9CdTzVrYFy7Y5R+TrE9HFUnsaXaGsJwOB/emByGPQEKrupz8CZFi9pkqPuAPwjN6Wonokv ChAewHXPUadcJmCTj78Oeg9uXR6yjpxyFjx3vdijQIYgi5TEGpeTQBymLANOYxYWYOjXk+ae dYuOYKR9nbPv+2zK9pwwQ2NXbUBystaGyQ== Message-ID: Date: Wed, 24 Jun 2020 19:32:23 +0100 MIME-Version: 1.0 In-Reply-To: <4650f0a7-64cf-831d-631c-243ea510fd38@intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [RFC] ethdev: add a field for rte_eth_rxq_info X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 6/24/2020 9:52 AM, Ferruh Yigit wrote: > On 6/24/2020 4:48 AM, Chengchang Tang wrote: >> >> On 2020/6/23 17:30, Andrew Rybchenko wrote: >>> On 6/23/20 9:48 AM, Chengchang Tang wrote: >>>> In common practice, PMD configure the rx_buf_size according to the data >>>> room size of the object in mempool. But in fact the final value is related >>>> to the specifications of hw, and its values will affect the number of >>>> fragments in recieving pkts. >>>> >>>> At present, we seem to have no way to espose relevant information to upper >>>> layer users. >>>> >>>> Add a field named rx_bufsize in rte_eth_rxq_info to indicate the buffer >>>> size used in recieving pkts for hw. >>>> >>> >>> I'm OK with the change in general. >>> I'm unsure which name to use: 'rx_buf_size' or 'rx_bursize', >>> since I found both 'min_rx_buf_size' and 'min_rx_bufsize' in >>> ethdev. >>> >>> I think it is important to update PMDs which provides the >>> information to fill the field in. >> >> My plan is to divide the subsequent series into two patches, >> one to modify rte_eth_rxq_info, and one to add our hns3 PMD >> implementation of rxq_info_get. Should i update all the PMDs >> that provide this information and test programs such as >> testpmd at the same time? > > Hi Chengchang, Andrew, > > No objection to the change, but it should be crystal clear what is added. These > are for PMD developers to implement and when it is not clear we end up having > different implementations and inconsistencies. > > There is already some confusion for the Rx packet size etc.. my concern is > adding more to it, here all we have is "size of RX buffer." comment, I think we > need more. cc'ed a few more people. Back to this favorite topic, how to configure/limit the packet size. Can you please help to have a common/correct understanding? I tried to clarify as much as I got it, any comment welcome. (I know it is long, please bare with me) The related config options I can see, 1) rte_eth_conf->rxmode->max_rx_pkt_len 2) rte_eth_dev_info->max_rx_pktlen 3) DEV_RX_OFFLOAD_JUMBO_FRAME 4) rte_eth_dev->data->mtu 5) DEV_RX_OFFLOAD_SCATTER 6) dev->data->scattered_rx 7) rte_eth_dev_info->min_mtu, rte_eth_dev_info->max_mtu 'mtu' (4): Both for Tx and Rx. The network layer payload length. Default value 'RTE_ETHER_MTU'. 'max_rx_pkt_len' (1): Only for Rx, maximum Rx frame length configured by application. 'max_rx_pktlen' (2): Device reported value on what maximum Rx frame length it can receive. Application shouldn't set Rx frame length more than this value. 'DEV_RX_OFFLOAD_JUMBO_FRAME' (3): Device Jumbo Frame capability. When not enabled the Rx frame length is 'MTU' + overhead When enabled Rx frame length is 'max_rx_pkt_len' 'DEV_RX_OFFLOAD_SCATTER' (5): Capability to scatter packet to multiple descriptor by device and driver converting this to chained mbuf. 'dev->data->scattered_rx' (6): The current status of driver scattered Rx, in device data mostly for PMD internal usage. 'rte_eth_dev_info->min_mtu' & 'rte_eth_dev_info->max_mtu' (7): minimum and maximum MTU values device supported. 'max_mtu' == 'max_rx_pkt_len' - L2_OVERHEAD. I can see two different limits, a) The Rx frame length limit that device can receive from wire. Any packet larger than this size will be dropped by device in an early stage. b) The Rx buffer length limit that received packets are written to. Device shouldn't DMA larger than reserved buffer size. If device supports scattered Rx to multiple descriptors, it can be possible to configure (a) > (b). Otherwise configuration have to be (b) >= (a). For example if the mbuf size is 2Kb and the device can receive up to 9000 bytes. Options are: - If device supports it, large packet will be scattered on multiple mbufs - or need to configure device Rx frame length to 2K (mbuf size) - or need to allocate mbuf big enough to get largest possible packet (9000) Issues I see: ------------- i) Although the code clearly says 'max_rx_pkt_len' is only valid when jumbo frames enabled, some drivers are taking it account always. ii) Some drivers enable 'scattered_rx' & 'jumbo frame' implicitly, without having 'DEV_RX_OFFLOAD_JUMBO_FRAME' or 'DEV_RX_OFFLOAD_SCATTER' requested by application. iii) Having both 'mtu' & 'max_rx_pkt_len' are confusing, although they are not exactly same thing they are related. Difference is MTU applies for Tx too, and L2 network layer overhead is not included. 'MTU' can be more interested by upper layers, 'max_rx_pkt_len' is more driver level information. And driver should know how to convert one to another. iv) 'max_rx_pkt_len' provided as part of 'rte_eth_dev_configure()' and there is no API to update it later. 'mtu' is not part of 'rte_eth_dev_configure()', it can only be updated later with specific API. But driver have to keep these two values consistent. v) 'max_rx_pktlen' & 'max_mtu' reports from driver are redundant information. Again they are not same thing, but correlated. Suggested changes: ----------------- Overall unify 'max_rx_pkt_len' & 'mtu' as much as possible, at first step: i) Make 'max_rx_pkt_len' independent from 'DEV_RX_OFFLOAD_JUMBO_FRAME', so 'max_rx_pkt_len' value will be always valid, jumbo frame enabled or not. ii) in '.dev_configure' convert 'max_rx_pkt_len' value to 'mtu' value, this will be only point 'max_rx_pkt_len' is used, after that point PMD will always use 'mtu' value. Even don't reflect 'rte_eth_dev_set_mtu()' changes to 'max_rx_pkt_len' anymore. iii) Don't make 'max_rx_pkt_len' a mandatory config option, let it be '0' by application, in that case 'rte_eth_dev_configure()' will set "'max_rx_pkt_len' = RTE_ETHER_MAX_LEN" if 'DEV_RX_OFFLOAD_JUMBO_FRAME' disabled "'max_rx_pkt_len' = 9000 if 'DEV_RX_OFFLOAD_JUMBO_FRAME' enabled iv) Allow implicit update of 'DEV_RX_OFFLOAD_JUMBO_FRAME' on MTU set, since setting a large MTU implies the jumbo frame request. And there is no harm to application. v) Do NOT allow implicit update of 'DEV_RX_OFFLOAD_SCATTER' on MTU set (when Rx frame length > Rx buffer length), since application may not be capable of parsing chained mbufs. Instead fails the MTU set in that case. [This can break some applications, relying on this implicit set.] Any comments? Additional details: ------------------- Behavior of some drivers: What igb & ixgbe does - Set Rx frame limit (a) using 'max_rx_pkt_len' (1) - Set Rx buffer limit (b) using mbuf data size - Enable Scattered Rx (5 & 6) if the Rx frame limit (a) bigger than Rx buffer limit (b) (even user not requested for it) What i40e does same as above, only differences - Return error if jumbo frame enabled and 'max_rx_pkt_len' < RTE_ETHER_MAX_LEN sfc: - Set Rx frame limit (a) - using 'max_rx_pkt_len' (1) when jumbo frame enabled - using 'mtu' when jumbo frame not enabled. - Set Rx buffer limit (b) using mbuf data size - If Rx frame limit (a) bigger than Rx buffer limit (b), and user not requested 'DEV_RX_OFFLOAD_SCATTER', return error. octeontx2: - Set Rx frame limit (a) using 'max_rx_pkt_len' (1). Implicitly enable jumbo frame based on 'max_rx_pkt_len'. - I can't able find how Rx buffer limit (b) set - Enable Scattered Rx (5) if the Rx frame limit (a) bigger than Rx buffer limit (b) (even user not requested for it). 'dev->data->scattered_rx' not set at all. > Adding a PMD implementation and testpmd updates helps to clarify the > intention/usage, so I suggest sending them as a single patch with this one. > > Updating all PMDs is a bigger ask and sometimes too hard because of lack of > knowledge on the internals of other PMDs, although this is causing feature gaps > time to time, we are not mandating this to developers, so please update as many > PMD as you can, that you are confident, rest should be done by their maintainers. > >>> >>>> Signed-off-by: Chengchang Tang >>> >>> Acked-by: Andrew Rybchenko >>> >>>> --- >>>> lib/librte_ethdev/rte_ethdev.h | 1 + >>>> 1 file changed, 1 insertion(+) >>>> >>>> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h >>>> index 0f6d053..82b7e98 100644 >>>> --- a/lib/librte_ethdev/rte_ethdev.h >>>> +++ b/lib/librte_ethdev/rte_ethdev.h >>>> @@ -1306,6 +1306,7 @@ struct rte_eth_rxq_info { >>>> struct rte_eth_rxconf conf; /**< queue config parameters. */ >>>> uint8_t scattered_rx; /**< scattered packets RX supported. */ >>>> uint16_t nb_desc; /**< configured number of RXDs. */ >>>> + uint16_t rx_bufsize; /**< size of RX buffer. */ >>>> } __rte_cache_min_aligned; >>>> >>>> /** >>>> -- >>>> 2.7.4 >>>> >>> >>> >>> . >>> >> >