From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 908E9A04B6; Mon, 12 Oct 2020 11:40:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D558B1D635; Mon, 12 Oct 2020 11:40:40 +0200 (CEST) Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) by dpdk.org (Postfix) with ESMTP id BD9EC1D62C for ; Mon, 12 Oct 2020 11:40:37 +0200 (CEST) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Mon, 12 Oct 2020 02:40:24 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 12 Oct 2020 09:40:34 +0000 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.47) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 12 Oct 2020 09:40:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CkOj/JilxpiJTaNQMC0xa7pRVIglERYWMQS6iq7FHPYEZxVJ4SgbKkEkMQ+GPJo371VHic5zq3bJWhopSpgwJOHX4XxqKXa9QcSzeDTSvYqOwEIyJ4xkn0iJT/ZhhJn9QESvja8Zkgc0w5+CR7HVFVSFvb3RuWoHi3P+NLhy8OV0+qBGNLw7PL37AePhZT+mMOWQJCUu/ex5Bgl21xCz/twXxz4pONAvM80vQndJrvYDT1G39O1Gsn1z7QseITO2KptrN1uIRE0/+dEWaxydcYLSSMF8SRJP+ULIBInZbAVHTuXzPFDAUd0CuvVIF2fDJAReXei6c3tLZhcp+hbFYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T6rH/M3QvPFJ/0ER+ehIoFl1hmKQEY7iq3FitygabRk=; b=B9061FTSJgC0DkWkVwWzdQQ7TbhR9LldTvbTiRsNpyvpO54/nEnHSVqJ1xyxM/XOk2dKDH92jdqYtnJMLuKz/u7EDJ2uaGk0ciDeJYtS7iBYppVqC88xGvNXXl6vtuCMrb/5ALLVravbg3Q94ILrBiPoXin9SrOnyWFddUbiWO51QyGxTqPD4N6ywiJ/PT22Il2RBBl/LBF2Vq73xYWqmcawjEU1waj4JiuESBbspAxfKfoSsg4afOS93fD0Rdks2z6P25KQuZ4OyVT/WGkwNP3+EB+yXXTcsL5jh2Rw6gw7ODlEluCkeXsPxy3TUogQxV5iG9kDHDCf4UHtlytI1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MWHPR12MB1360.namprd12.prod.outlook.com (2603:10b6:300:12::7) by MWHPR1201MB0253.namprd12.prod.outlook.com (2603:10b6:301:52::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.21; Mon, 12 Oct 2020 09:40:32 +0000 Received: from MWHPR12MB1360.namprd12.prod.outlook.com ([fe80::711e:ec6f:ba28:d3d0]) by MWHPR12MB1360.namprd12.prod.outlook.com ([fe80::711e:ec6f:ba28:d3d0%5]) with mapi id 15.20.3455.030; Mon, 12 Oct 2020 09:40:31 +0000 From: Slava Ovsiienko To: NBU-Contact-Thomas Monjalon CC: "dev@dpdk.org" , "stephen@networkplumber.org" , "ferruh.yigit@intel.com" , "olivier.matz@6wind.com" , "jerinjacobk@gmail.com" , "maxime.coquelin@redhat.com" , "david.marchand@redhat.com" , "arybchenko@solarflare.com" Thread-Topic: [dpdk-dev] [PATCH v2 1/9] ethdev: introduce Rx buffer split Thread-Index: AQHWnLuLrf8Js/AyR0+XVog1O1b9pamS/zUAgAC8Y6A= Date: Mon, 12 Oct 2020 09:40:31 +0000 Message-ID: References: <1602083215-22921-1-git-send-email-viacheslavo@nvidia.com> <1602083215-22921-2-git-send-email-viacheslavo@nvidia.com> <5451712.xXqzTqBGid@thomas> In-Reply-To: <5451712.xXqzTqBGid@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c1260e96-c747-4244-a8c5-08d86e92dc96 x-ms-traffictypediagnostic: MWHPR1201MB0253: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4oqoHkviHmUxxzC6vb2IyL/qm1RaZ+l9ihclZINsQEevKnxSbuoMKLXMNDQOgaD8qotM2QzE/H6zx8t4lzf0G7SznZiJP8UdcoNuDEcxNKJEUM0akkDKc6rYoLoD5/xOQq9Ns1SpLRl11Xj/0EPUCEPUbja8Gl1vc5UW3qYEI89ubbDEjYLozIdEh5XKF3JeZJw2dwqD+jI1LMZnYOq0WNNW/bB3RnMlzW+BTU0DvyGrdfvt5b5n9uGP8Jt0TcZARa68tai6CJSLigAY4iQmPTtLUj7jof1/7OeVyrtxqWTTDVhW4/ayineD8QI5SIcr9gMxSjD6Vh4CU2Vcoqx/Ng== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR12MB1360.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(366004)(39860400002)(376002)(52536014)(66946007)(26005)(76116006)(83380400001)(2906002)(5660300002)(4326008)(7696005)(8676002)(6916009)(71200400001)(9686003)(86362001)(55016002)(66476007)(186003)(66556008)(316002)(64756008)(6506007)(53546011)(478600001)(8936002)(33656002)(66446008)(54906003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: uljktTovWvPIgPqhxn1Td/bpP6RNfC/9Ij3u9y2UwixXvq8Dkhz69EEIqhbiIEs7Kt1CcX7fCr7e1RTXYnbiigOx8TreZL2BQ+LmqfBONxHQvfShLvCe+LJFYXNQOZe0tfpdLE/a3jcdrYPiKJGcyz/ER/J5Hig9Xk+syVL7op8IDrr5LwwuF6fEGaKWvEqD2BmceL6gkr3rHy8Q+aFpPXgBtgem9yRudXEZB3eFAzJ4Yb99k9GOYkmcS7L1s8uCH5kfdUgXNPCej+MPcHldZkTBKH0Bf8YaYznJwI2eKPne1BeO0u9xkhYhbfIH8s2p8P23qcVEXy5DJK+bpjg8imBlhOFWMEZNuVr0bIAUaF/Reywoz8y6MRax7QjZ234E2qN+l0C5xRANwU8zq1DLfdGmz+Ujdc1QFJ5nYoHwrzv4O29Y2euAeS85JnbAbPiG/IaMQYpSpnWzjlo2Lem/HaEskFitF57j64wfYFCRkpWamcUUEOSoxav2i+zbKdgRy+gREHIpuCrALsRybkXCRTbq/dfgh4tYkXtHGa9+ZyaA7Bpb+YBJ1gZPrvfWDqlEm2Oaie6+bScafYLBiaX2IC/MjQycSs6sODomZ4Qx6Sf+XWsrDR8RLweUBD0+0ZEtqRjJhM3ue/8a2FKgzPnWew== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR12MB1360.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c1260e96-c747-4244-a8c5-08d86e92dc96 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Oct 2020 09:40:31.5488 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zjmtLK9R5HF/jPVpMAc0QdSdlox93Cq+gOS2BuBwCT+uXx/T9G8mVoAiw9HVEsLa6AMJgNIQhlz0bHXNAISTUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0253 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1602495624; bh=T6rH/M3QvPFJ/0ER+ehIoFl1hmKQEY7iq3FitygabRk=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:x-ms-exchange-transport-forked: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=UWwF64K3dupwk8qYhXuAgKL+XL3KZjRyuXoD5NAkva9KbqgjqcbzgJi4Q9QOtxppj HVyC7DdLD/vIrlfJVuNWBHrwkhqzxELa1mbMsSeYnY65zjfa4cWS55SrkhnRHN66Io MbI5YOJF8nNiNVm+tgivNToXKpDOnM4AwBglmsers0wJF4WOW+gdY7vc1rnYk2ZZlt DtrDdqJXL2wA8eCs5n6lr6pDTVvk3vdh1q30jqsnzyjwwY/yvXIKvuzihZURlcCg6p RsWK2PMxXcmlvmo3uUeQ8OaVhymZDSbNpmM1XPBtU/4WA0rfI5rqLJAIwvQ5iZ4FKP OpUaMkDGHsVZA== Subject: Re: [dpdk-dev] [PATCH v2 1/9] ethdev: introduce Rx buffer split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, Thomas Thank you for the comments, please, see my answers below. > -----Original Message----- > From: Thomas Monjalon > Sent: Monday, October 12, 2020 1:18 > To: Slava Ovsiienko > Cc: dev@dpdk.org; stephen@networkplumber.org; ferruh.yigit@intel.com; > olivier.matz@6wind.com; jerinjacobk@gmail.com; > maxime.coquelin@redhat.com; david.marchand@redhat.com; > arybchenko@solarflare.com > Subject: Re: [dpdk-dev] [PATCH v2 1/9] ethdev: introduce Rx buffer split >=20 > 07/10/2020 17:06, Viacheslav Ovsiienko: > > The DPDK datapath in the transmit direction is very flexible. > > An application can build the multi-segment packet and manages almost > > all data aspects - the memory pools where segments are allocated from, > > the segment lengths, the memory attributes like external buffers, > > registered for DMA, etc. > > > > In the receiving direction, the datapath is much less flexible, an > > application can only specify the memory pool to configure the > > receiving queue and nothing more. In order to extend receiving > > datapath capabilities it is proposed to add the way to provide > > extended information how to split the packets being received. > > > > The following structure is introduced to specify the Rx packet > > segment: > > > > struct rte_eth_rxseg { > > struct rte_mempool *mp; /* memory pools to allocate segment from */ > > uint16_t length; /* segment maximal data length */ >=20 > The "length" parameter is configuring a split point. > Worth to note in the comment I think. OK, got it. >=20 > > uint16_t offset; /* data offset from beginning of mbuf data buffer > > */ >=20 > Is it replacing RTE_PKTMBUF_HEADROOM? >=20 Actually adding to HEAD_ROOM. We should keep HEAD_ROOM intact, so actual data offset in the firtst mbuf must be the sum HEAD_ROOM + offset= . mlx5 PMD Imlementation follows this approach, documentation will be updated= in v3. > > uint32_t reserved; /* reserved field */ }; > > > > The new routine rte_eth_rx_queue_setup_ex() is introduced to setup the > > given Rx queue using the new extended Rx packet segment > > description: > > > > int > > rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, > > uint16_t nb_rx_desc, unsigned int socket_id, > > const struct rte_eth_rxconf *rx_conf, > > const struct rte_eth_rxseg *rx_seg, > > uint16_t n_seg) >=20 > An alternative name for this function: > rte_eth_rxseg_queue_setup M-m-m... Routine name follows patter object_verb: rx_queue is an object, setup is an action. rxseg_queue is not an object. What about "rte_eth_rx_queue_setup_seg"? >=20 > > This routine presents the two new parameters: > > rx_seg - pointer the array of segment descriptions, each element > > describes the memory pool, maximal data length, initial > > data offset from the beginning of data buffer in mbuf > > n_seg - number of elements in the array >=20 > Not clear why we need an array. > I suggest writing here that each segment of the same packet can have > different properties, the array representing the full packet. OK, will write. >=20 > > The new offload flag DEV_RX_OFFLOAD_BUFFER_SPLIT in device >=20 > The name should start with RTE_ prefix. It is an existing pattern for DEV_RX_OFFLOAD_xxxx, no RTE_ for the case. >=20 > > capabilities is introduced to present the way for PMD to report to > > application about supporting Rx packet split to configurable segments. > > Prior invoking the rte_eth_rx_queue_setup_ex() routine application > > should check DEV_RX_OFFLOAD_BUFFER_SPLIT flag. > > > > If the Rx queue is configured with new routine the packets being > > received will be split into multiple segments pushed to the mbufs with > > specified attributes. The PMD will allocate the first mbuf from the > > pool specified in the first segment descriptor and puts the data > > staring at specified offset in the allocated mbuf data buffer. If > > packet length exceeds the specified segment length the next mbuf will > > be allocated according to the next segment descriptor (if any) and > > data will be put in its data buffer at specified offset and not > > exceeding specified length. If there is no next descriptor the next > > mbuf will be allocated and filled in the same way (from the same pool > > and with the same buffer offset/length) as the current one. > > > > For example, let's suppose we configured the Rx queue with the > > following segments: > > seg0 - pool0, len0=3D14B, off0=3DRTE_PKTMBUF_HEADROOM > > seg1 - pool1, len1=3D20B, off1=3D0B > > seg2 - pool2, len2=3D20B, off2=3D0B > > seg3 - pool3, len3=3D512B, off3=3D0B > > > > The packet 46 bytes long will look like the following: > > seg0 - 14B long @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 > > seg1 - 20B long @ 0 in mbuf from pool1 > > seg2 - 12B long @ 0 in mbuf from pool2 > > > > The packet 1500 bytes long will look like the following: > > seg0 - 14B @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 > > seg1 - 20B @ 0 in mbuf from pool1 > > seg2 - 20B @ 0 in mbuf from pool2 > > seg3 - 512B @ 0 in mbuf from pool3 > > seg4 - 512B @ 0 in mbuf from pool3 > > seg5 - 422B @ 0 in mbuf from pool3 > > > > The offload DEV_RX_OFFLOAD_SCATTER must be present and configured > to > > support new buffer split feature (if n_seg is greater than one). > > > > The new approach would allow splitting the ingress packets into > > multiple parts pushed to the memory with different attributes. > > For example, the packet headers can be pushed to the embedded data > > buffers within mbufs and the application data into the external > > buffers attached to mbufs allocated from the different memory pools. > > The memory attributes for the split parts may differ either - for > > example the application data may be pushed into the external memory > > located on the dedicated physical device, say GPU or NVMe. This would > > improve the DPDK receiving datapath flexibility with preserving > > compatibility with existing API. > > > > Also, the proposed segment description might be used to specify Rx > > packet split for some other features. For example, provide the way to > > specify the extra memory pool for the Header Split feature of some > > Intel PMD. >=20 > I don't understand what you are referring in this last paragraph. > I think explanation above is enough to demonstrate the flexibility. >=20 Just noted the segment description is common thing and could be promoted to be used in some other features.=20 > > Signed-off-by: Viacheslav Ovsiienko >=20 > Thank you, I like this feature. > More minor comments below. >=20 > [...] > > +* **Introduced extended buffer description for receiving.** >=20 > Rewording: > Introduced extended setup of Rx queue OK, sounds better. >=20 > > + * Added extended Rx queue setup routine > > + * Added description for Rx segment sizes >=20 > not only "sizes", but also offset and mempool. >=20 > > + * Added capability to specify the memory pool for each segment >=20 > This one can be merged with the above, or offset should be added. >=20 > [...] > The doxygen comment is missing here. Yes, thank you. Also noted that, updating. >=20 > > +int rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, > > + uint16_t nb_rx_desc, unsigned int socket_id, > > + const struct rte_eth_rxconf *rx_conf, > > + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg); >=20 > This new function should be experimental and it should be added to the > .map file. >=20 OK. With best regards, Slava