From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 51D26A04DB; Thu, 15 Oct 2020 13:09:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1CC2D1E31C; Thu, 15 Oct 2020 13:09:51 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by dpdk.org (Postfix) with ESMTP id 3FC201DEFA for ; Thu, 15 Oct 2020 13:09:48 +0200 (CEST) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id E85937F56C; Thu, 15 Oct 2020 14:09:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru E85937F56C DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1602760187; bh=e594iPj58uD7l+TkgASd6qTE2sjz4rpq1vLZzuU9bdU=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=d7wzC9rmtG2krcYR9tXeWWGNXlw73f8wqg1z5Mza2l0UsP2xdSY8zBiz3yFT5pEiu GFccWNllkGg3d2unCza4fxwxO2hNwiqdo9M5uO7SGcFiyqT81ICRHBZ1JL6K1/igWh LB6eo66LL1zXHxPFggehaWB3MJaFcKMzp3DhGVo0= To: Slava Ovsiienko , Jerin Jacob Cc: dpdk-dev , NBU-Contact-Thomas Monjalon , Stephen Hemminger , Ferruh Yigit , Olivier Matz , Maxime Coquelin , David Marchand , Andrew Rybchenko References: <1602699122-15737-1-git-send-email-viacheslavo@nvidia.com> <1602699122-15737-2-git-send-email-viacheslavo@nvidia.com> <663a2ea8-cb32-9960-75d1-2a4854dcc0ff@oktetlabs.ru> From: Andrew Rybchenko Organization: OKTET Labs Message-ID: <6e190086-a859-2e3f-bf76-d56b193fd906@oktetlabs.ru> Date: Thu, 15 Oct 2020 14:09:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v6 1/6] ethdev: introduce Rx buffer split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/15/20 1:34 PM, Slava Ovsiienko wrote: > Hi, Andrew > >> -----Original Message----- >> From: Andrew Rybchenko >> Sent: Thursday, October 15, 2020 12:49 >> To: Slava Ovsiienko ; Jerin Jacob >> >> Cc: dpdk-dev ; NBU-Contact-Thomas Monjalon >> ; Stephen Hemminger >> ; Ferruh Yigit ; >> Olivier Matz ; Maxime Coquelin >> ; David Marchand >> ; Andrew Rybchenko >> >> Subject: Re: [dpdk-dev] [PATCH v6 1/6] ethdev: introduce Rx buffer split >> >> On 10/15/20 10:43 AM, Slava Ovsiienko wrote: >>> Hi, Jerin >>> >>>> -----Original Message----- >>>> From: Jerin Jacob >>>> Sent: Wednesday, October 14, 2020 21:57 >>>> To: Slava Ovsiienko >>>> Cc: dpdk-dev ; NBU-Contact-Thomas Monjalon >>>> ; Stephen Hemminger >>>> ; Ferruh Yigit ; >>>> Olivier Matz ; Maxime Coquelin >>>> ; David Marchand >>>> ; Andrew Rybchenko >>>> >>>> Subject: Re: [PATCH v6 1/6] ethdev: introduce Rx buffer split >>>> >>>> On Wed, Oct 14, 2020 at 11:42 PM Viacheslav Ovsiienko >>>> wrote: >>>>> >>>>> The DPDK datapath in the transmit direction is very flexible. >>>>> An application can build the multi-segment packet and manages almost >>>>> all data aspects - the memory pools where segments are allocated >>>>> from, the segment lengths, the memory attributes like external >>>>> buffers, registered for DMA, etc. >>>>> >>> >>> [..snip..] >>> >>>>> For example, let's suppose we configured the Rx queue with the >>>>> following segments: >>>>> seg0 - pool0, len0=14B, off0=2 >>>>> seg1 - pool1, len1=20B, off1=128B >>>>> seg2 - pool2, len2=20B, off2=0B >>>>> seg3 - pool3, len3=512B, off3=0B >>>> >>>> >>>> Sorry for chime in late. This API lookout looks good to me. >>>> But, I am wondering how the application can know the capability or >>>> "limits" of struct rte_eth_rxseg structure for the specific PMD. The >>>> other descriptor limit, it's being exposed with struct >>>> rte_eth_dev_info::rx_desc_lim; If PMD can support a specific pattern >>>> rather than returning the blanket error, the application should know the >> limit. >>>> IMO, it is better to add >>>> struct rte_eth_rxseg *rxsegs; >>>> unint16_t nb_max_rxsegs >>>> in rte_eth_dev_info structure to express the capablity. >>>> Where the en and offset can define the max offset. >>>> >>>> Thoughts? >>> >>> Moreover, there might be implied a lot of various limitations - >>> offsets might be not supported at all or have some requirements for >>> alignment, the similar requirements might be applied to segment size >>> (say, ask for some granularity). Currently it is not obvious how to >>> report all nuances, and it is supposed the limitations of this kind must be >> documented in PMD chapter. As for mlx5 - it has no special limitations besides >> common requirements to the regular segments. >>> >>> One more point - the split feature might be considered as just one of >>> possible cases of using these segment descriptions, other features might >> impose other (unknown for now) limitations. >>> If we see some of the features of such kind or other PMDs adopts the >>> split feature - we'll try to find the common root and consider the way how to >> report it. >> >> At least there are few simple limitations which are easy to >> express: >> 1. Maximum number of segments > We have scatter capability and we do not report the maximal number of segments, > it is on PMD own. We could add the field to the rte_eth_dev_info, but not sure > whether we have something special to report there even for mlx5 case. There is always a limitation in programming and HW. Nothing is unlimited. Limits could be high, but still exist. Number of descriptors? Width of field in HW interface? Maximum length of the config message to HW? All above could limit it directly or indirectly. >> 2. Possibility to use the last segment many times if required >> (I was suggesting to use scatter for it, but you rejected >> the idea - may be time to reconsider :) ) > > Mmm, sorry I do not follow, it might be I did not understand/missed your idea. > Some of the last segment attributes are used multiple times to scatter the rest > of the data in fashion very close to the existing scattering approach - at least, > pool and buffer size from this pool are used. The beginning of the packet > scattered according to the new descriptions, the rest of the packet - > according to the existing regular scattering with pool settings from > the last segment description. I believe that the possibility to split into a fixed segments (BUFFER_SPLIT) and possibility to use a mempool (just mp or last segment) many times if a packet does not fit (SCATTER) it is *different* features. I can easily imagine HW which could do BUFFER_SPLIT to fixed segments, but cannot use the last segment many times (i.e. no classical SCATTER). > > 3. Maximum offset >> Frankly speaking I'm not sure why it cannot be handled on >> PMD level (i.e. provide descriptors with offset taken into >> account or guarantee that HW mempool objects initialized >> correctly with required headroom). May be in some corner >> cases when the same HW mempool is shared by various >> segments with different offset requirements. > > HW offsets are beyond the feature scope, the offsets in the segment > description is supposed to be added to the native pool offsets (if any). Are you saying that offsets are not passed to HW and just handled by PMD to provide correct IOVA addresses to put data to? If so, it is an implementation detail which is specific to mlx5. If so, no specific limitations except data room, size and offset consistency. But it could be passed to a HW and it could be, for example, just 8 bits for the value. > >> 4. Offset alignment >> 5. Maximum/minimum length of a segment >> 6. Length alignment > In which form? Mask of lsbs ? 0 means no limitations ? log2, i.e. 0 => 1 (no limitations) 1 => 2 (even only), 6 => 64 (64-byte cache line aligned) etc. > >> >> I realize that 3, 4 and 5 could be per segment number. >> If it is really that complex, report common denominator which is guaranteed to >> work. If we have no checks on ethdev layer, application can ignore it if it knows >> better. > > Currently it is not clear at all what kind of limitations should be reported, > we could include all of mentioned/proposed ones, and no one will report there - > mlx5 has no any reasonable limitations to report for now. > > Should we reserve some pointer field in the rte_eth_dev_info to report > the limitations? (Limitation description should contain variable size array, > depending on the number of segments, so pointer seems to be appropriate). > It would allow us to avoid ABI break, and present the limitation structure once it is defined. I will let other ethdev maintainers to make a decision here. My vote would be to report limitations mentioned above. It looks like Jerin is also interested in limitations reporting. Not sure if my form looks OK or no.