From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EC7AC45588; Thu, 4 Jul 2024 13:08:17 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CCB7D40280; Thu, 4 Jul 2024 13:08:17 +0200 (CEST) Received: from office2.cesnet.cz (office2.cesnet.cz [78.128.248.237]) by mails.dpdk.org (Postfix) with ESMTP id 7AC5E4025C for ; Thu, 4 Jul 2024 13:08:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cesnet.cz; s=office2-2020; t=1720091295; bh=QQ0f98fLa7qVazlsZFOJ2TWe2bNQTnrwpynrDbHdyCU=; h=Date:Subject:From:To:Cc:References:In-Reply-To; b=Qdhx7EIHBVUW/z5XXu3l4BWIxQDRP9WfySLsamV+b5hAxtkSlnMrWFt0y6dia2NOF r31F+gIrGu57gkqNjQqU7M9kgDllRRzHTMJeiWDQQAGLfYcFyQkSKHnjYJRK13DRQD 76Bl1sBx4AblMjoyiGMlkQRZn8gfHLi/iuHRZcJ8FN3UsBViy0zGBO70WJnro1db5M 2rBGOQTARiBgfPzWZD8uv/CYMwQqs/ZO80KZzFUgt0DyNFWziNdN9czC/9fW4ywv7w 47LVYTF6Q9S5RLyzzxBo98vboW0XQxnxcZmbuZymcz9nnrqSpcE6VJLpZHJ9RM/yw0 iTWmpg/v20PoQ== Received: from [IPV6:2a01:c846:1282:6700:7d1a:26f4:e4a1:39f5] (unknown [IPv6:2a01:c846:1282:6700:7d1a:26f4:e4a1:39f5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by office2.cesnet.cz (Postfix) with ESMTPSA id 655981180081; Thu, 4 Jul 2024 13:08:12 +0200 (CEST) Message-ID: <95896e0b-648c-40ee-986e-2f4c6d0bb29a@cesnet.cz> Date: Thu, 4 Jul 2024 13:08:10 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Hairpin Queues Throughput ConnectX-6 From: =?UTF-8?Q?M=C3=A1rio_Kuka?= To: dev@dpdk.org Cc: orika@nvidia.com, bingz@nvidia.com, viktorin@cesnet.cz, Dmitry Kozlyuk References: <3d746dbc-330e-403f-b87f-bf495cac3437@cesnet.cz> <20240625032224.45b65339@sovereign> <82d8f67c-3b0b-46c2-a94b-8457d0c602c2@cesnet.cz> Content-Language: en-US, sk-SK In-Reply-To: <82d8f67c-3b0b-46c2-a94b-8457d0c602c2@cesnet.cz> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="------------ms040406050304030204000308" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a cryptographically signed message in MIME format. --------------ms040406050304030204000308 Content-Type: multipart/alternative; boundary="------------6Tw3mqV5enY9qt2YU7l4oIv6" --------------6Tw3mqV5enY9qt2YU7l4oIv6 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 SGVsbG8NCg0KSSBkaWQgZXhwZXJpbWVudHMgd2hlcmUgSSBzZW50IHBhY2tldHMgdG8gdGhl IGhhaXJwaW4gcXVldWVzIGFuZCB0aGUgQ1BVIA0KcXVldWUgYXQgdGhlIHNhbWUgdGltZS4N CkR1cmluZyB0ZXN0aW5nLCBJIGZvdW5kIHRoYXQgd2hlbiB0aGUgQ1BVIHF1ZXVlIGlzIG92 ZXJsb2FkZWQgdG9vIG11Y2gsIA0KdGhlIGhhaXJwaW4gcXVldWVzIGFsc28gYmVnaW4gdG8g ZHJvcCBwYWNrZXRzLg0KDQpFeGFtcGxlIDE6DQpTZW5kaW5nIDEwIEdicHMgdG8gaGFpcnBp biBxdWV1ZXMNClJlc3VsdGluZyB0aHJvdWdocHV0IGlzIDEwIEdicHMNCkV4cGVjdGVkIHJl c3VsdA0KDQpFeGFtcGxlIDI6DQpTZW5kaW5nIDIwIEdicHMgdG8gQ1BVIHF1ZXVlDQpSZXN1 bHRpbmcgdGhyb3VnaHB1dCBpcyAxMSBHYnBzICg5IEdicHMgZHJvcCkNCkV4cGVjdGVkIHJl c3VsdA0KDQpFeGFtcGxlIDM6DQpTZW5kaW5nIDEwIEdicHMgdG8gaGFpcnBpbiBxdWV1ZXMg YW5kIDIwIEdicHMgdG8gQ1BVIHF1ZXVlDQpSZXN1bHRpbmcgdGhyb3VnaHB1dCBpcyAyMUdi cHMsIDEwIEdicHMgKHplcm8gcGFja2V0IGRyb3ApIGZyb20gaGFpcnBpbiANCisgMTEgR2Jw cyBmcm9tIENQVQ0KRXhwZWN0ZWQgcmVzdWx0DQoNCkV4YW1wbGUgNDoNClNlbmRpbmcgMTAg R2JwcyB0byBoYWlycGluIHF1ZXVlcyBhbmQgNTAgR2JwcyB0byBDUFUgcXVldWUNClJlc3Vs dGluZyB0aHJvdWdocHV0IGlzIDE2IEdicHMsIDVHYnBzICg1MCUrIHBhY2tldCBkcm9wKSBm cm9tIGhhaXJwaW4gKyANCjExR2JwcyBmcm9tIENQVSwNClVuZXhwZWN0ZWQgcmVzdWx0Li4u DQoNCkV4cGVyaW1lbnRzIHNldHVwOg0Kc3VkbyBtbHhjb25maWcgLXkgLWQgMDAwMDpjNDow MC4wIHNldCBNRU1JQ19TSVpFX0xJTUlUPTAgDQpIQUlSUElOX0RBVEFfQlVGRkVSX0xPQ0s9 MQ0Kc3VkbyBtbHhmd3Jlc2V0IC15IC1kIDAwMDA6YzQ6MDAuMCByZXNldA0Kc3VkbyBkcGRr LXRlc3RwbWQgLWwgMC0xIC1uIDQgLWEgMDAwMDpjNDowMC4wLGhwX2J1Zl9sb2dfc3o9MTMg LS0gDQotLXJ4cT0xIC0tdHhxPTEgLS1oYWlycGlucT0xMiAtLWhhaXJwaW4tbW9kZT0weDEx MTAgLWkNCiDCoMKgwqAgZmxvdyBjcmVhdGUgMCBpbmdyZXNzIHBhdHRlcm4gZXRoIHNyYyBp cyAwMDoxMDo5NDowMDowMDowMiAvIGVuZCANCmFjdGlvbnMgcXVldWUgaW5kZXggMCAvIGVu ZA0KIMKgwqDCoCBmbG93IGNyZWF0ZSAwIGluZ3Jlc3MgcGF0dGVybiBldGggc3JjIGlzIDAw OjEwOjk0OjAwOjAwOjAzIC8gZW5kIA0KYWN0aW9ucyByc3MgcXVldWVzIDEgMiAzIDQgNSA2 IDcgOMKgIGVuZCAvIGVuZA0KDQpTbyBJIGNhbid0IGFjaGlldmUgbXkgZ29hbDogdHJhZmZp YyBmcm9tIHRoZSBoYWlycGluIHF1ZXVlcyBpcyBub3QgDQpkcm9wcGVkIGlmIHRoZSBDUFUg cXVldWUgaXMgb3ZlcmxvYWRlZC4NCkFueSBpZGVhIGhvdyB0byBhY2hpZXZlIHRoaXMgaW4g ZXhhbXBsZSA0Pw0KV2hhdCBpcyB0aGUgcHJvYmxlbSwgZnVsbCBwYWNrZXQgYnVmZmVycy9t ZW1vcnkgaW4gdGhlIGRldmljZSB0aGF0IGFyZSANCnNoYXJlZCBiZXR3ZWVuIHRoZSBoYWly cGluIGFuZCBDUFUgcXVldWVzPw0KDQpBbnkgZ3VpZGFuY2Ugb3Igc3VnZ2VzdGlvbnMgb24g aG93IHRvIGFjaGlldmUgdGhpcyB3b3VsZCBiZSBncmVhdGx5IA0KYXBwcmVjaWF0ZWQuDQoN Ck3DoXJpbw0KDQpPbiAyNy8wNi8yMDI0IDEzOjQyLCBNw6FyaW8gS3VrYSB3cm90ZToNCj4g SGkgRG1pdHJ5LA0KPg0KPiBUaGFuayB5b3UgZm9yIHlvdXIgaGVscGZ1bCByZXBseS4NCj4+ IFRyeSBlbmFibGluZyAiRXhwbGljaXQgVHggcnVsZSIgbW9kZSBpZiBwb3NzaWJsZS4NCj4+ IEkgd2FzIGFibGUgdG8gYWNoaWV2ZSAxMzcgTXBwcyBAIDY0QiB3aXRoIHRoZSBmb2xsb3dp bmcgY29tbWFuZDoNCj4+DQo+PiBkcGRrLXRlc3RwbWQgLWEgMjE6MDAuMCAtYSBjMTowMC4w IC0taW4tbWVtb3J5IC0tIFwNCj4+ICAgICAgLWkgLS1yeHE9MSAtLXR4cT0xIC0taGFpcnBp bnE9OCAtLWhhaXJwaW4tbW9kZT0weDEwDQo+DQo+IEJhc2VkIG8gdGhpcyBJIHdhcyBhYmxl IHRvIGFjaGlldmUgMTQyIE1wcHMoOTYuMDggR2JwcykgQCA2NEIgd2l0aCB0aGUgZm9sbG93 aW5nIGNvbW1hbmQ6DQo+DQo+IHN1ZG8gZHBkay10ZXN0cG1kIC1sIDAtMSAtbiA0IC1hIDAw MDA6YzQ6MDAuMCxocF9idWZfbG9nX3N6PTEzIFwNCj4gICAgICAtLWluLW1lbW9yeSAtLSAt LXJ4cT0xIC0tdHhxPTEgLS1oYWlycGlucT0xMiAtLWhhaXJwaW4tbW9kZT0weDEwIC1pDQo+ ICAgICAgDQo+IGZsb3cgY3JlYXRlIDAgaW5ncmVzcyBwYXR0ZXJuIGV0aCBzcmMgaXMgMDA6 MTA6OTQ6MDA6MDA6MDIgLyBlbmQgYWN0aW9ucyByc3MgcXVldWVzIDEgMiAzIDQgNSA2IDcg OCA5IDEwIDExIDEyIGVuZCAvIGVuZA0KPg0KPiBBbG1vc3QgZnVsbCBzcGVlZCA6KS4NCj4g QW55IG90aGVyIHZhbHVlIG9mICJocF9idWZfbG9nX3N6IiBvciBtb3JlIHF1ZXVlcyBkb2Vz IG5vdCBnZXQgYmV0dGVyIHJlc3VsdHMsIGJ1dCBpbnN0ZWFkIG1ha2VzIHRoZW0gd29yc2Uu DQo+DQo+PiBSeFEgcGlubmVkIGluIGRldmljZSBtZW1vcnkgcmVxdWlyZXMgZmlybXdhcmUg Y29uZmlndXJhdGlvbiBbMV06DQo+Pg0KPj4gbWx4Y29uZmlnIC15IC1kICRwY2lfYWRkciBz ZXQgTUVNSUNfU0laRV9MSU1JVD0wIEhBSVJQSU5fREFUQV9CVUZGRVJfTE9DSz0xDQo+PiBt bHhmd3Jlc2V0IC15IC1kICRwY2lfYWRkciByZXNldA0KPj4NCj4+IFsxXTpodHRwczovL2Rv Yy5kcGRrLm9yZy9ndWlkZXMvcGxhdGZvcm0vbWx4NS5odG1sP2hpZ2hsaWdodD1oYWlycGlu X2RhdGFfYnVmZmVyX2xvY2sNCj4+DQo+PiBIb3dldmVyLCBwaW5uZWQgUnhRIGRpZG4ndCBp bXByb3ZlIGFueXRoaW5nIGZvciBtZS4NCj4NCj4gSSB0cmllZCBpdCwgYnV0IGl0IGRpZG4n dCBpbXByb3ZlIGFueXRoaW5nIGZvciBtZSBlaXRoZXIuDQo+DQo+IE3DoXJpbw0KPg0KPiBP biAyNS8wNi8yMDI0IDAyOjIyLMKgIEtvemx5dWsgd3JvdGU6DQo+PiBIaSBNw6FyaW8sDQo+ Pg0KPj4gMjAyNC0wNi0xOSAwODo0NSAoVVRDKzAyMDApLCBNw6FyaW8gS3VrYToNCj4+PiBI ZWxsbywNCj4+Pg0KPj4+IEkgd2FudCB0byB1c2UgaGFpcnBpbiBxdWV1ZXMgdG8gZm9yd2Fy ZCBoaWdoIHByaW9yaXR5IHRyYWZmaWMgKHN1Y2ggYXMNCj4+PiBMQUNQKS4NCj4+PiBNeSBn b2FsIGlzIHRvIGVuc3VyZSB0aGF0IHRoaXMgdHJhZmZpYyBpcyBub3QgZHJvcHBlZCBpbiBj YXNlIHRoZQ0KPj4+IHNvZnR3YXJlIHBpcGVsaW5lIGlzIG92ZXJ3aGVsbWVkLg0KPj4+IEJ1 dCBkdXJpbmcgdGVzdGluZyB3aXRoIGRwZGstdGVzdHBtZCBJIGNhbid0IGFjaGlldmUgZnVs bCB0aHJvdWdocHV0IGZvcg0KPj4+IGhhaXJwaW4gcXVldWVzLg0KPj4gRm9yIG1haW50YWlu ZXJzOiBJJ2QgbGlrZSB0byBleHByZXNzIGludGVyZXN0IGluIHRoaXMgdXNlIGNhc2UgdG9v Lg0KPj4NCj4+PiBUaGUgYmVzdCByZXN1bHQgSSBoYXZlIGJlZW4gYWJsZSB0byBhY2hpZXZl IGZvciA2NEIgcGFja2V0cyBpcyA4MyBHYnBzDQo+Pj4gaW4gdGhpcyBjb25maWd1cmF0aW9u Og0KPj4+ICQgc3VkbyBkcGRrLXRlc3RwbWQgLWwgMC0xIC1uIDQgLWEgMDAwMDoxNzowMC4w LGhwX2J1Zl9sb2dfc3o9MTkgLS0NCj4+PiAtLXJ4cT0xIC0tdHhxPTEgLS1yeGQ9NDA5NiAt LXR4ZD00MDk2IC0taGFpcnBpbnE9Mg0KPj4+IHRlc3RwbWQ+IGZsb3cgY3JlYXRlIDAgaW5n cmVzcyBwYXR0ZXJuIGV0aCBzcmMgaXMgMDA6MTA6OTQ6MDA6MDA6MDMgLw0KPj4+IGVuZCBh Y3Rpb25zIHJzcyBxdWV1ZXMgMSAyIGVuZCAvIGVuZA0KPj4gVHJ5IGVuYWJsaW5nICJFeHBs aWNpdCBUeCBydWxlIiBtb2RlIGlmIHBvc3NpYmxlLg0KPj4gSSB3YXMgYWJsZSB0byBhY2hp ZXZlIDEzNyBNcHBzIEAgNjRCIHdpdGggdGhlIGZvbGxvd2luZyBjb21tYW5kOg0KPj4NCj4+ IGRwZGstdGVzdHBtZCAtYSAyMTowMC4wIC1hIGMxOjAwLjAgLS1pbi1tZW1vcnkgLS0gXA0K Pj4gICAgICAtaSAtLXJ4cT0xIC0tdHhxPTEgLS1oYWlycGlucT04IC0taGFpcnBpbi1tb2Rl PTB4MTANCj4+DQo+PiBZb3UgbWlnaHQgZ2V0IGV2ZW4gYmV0dGVyIHNwZWVkLCBiZWNhdXNl IG15IGZsb3cgcnVsZXMgd2VyZSBtb3JlIGNvbXBsaWNhdGVkDQo+PiAoUlRFIEZsb3cgYmFz ZWQgInJvdXRlciBvbi1hLXN0aWNrIik6DQo+Pg0KPj4gZmxvdyBjcmVhdGUgMCBpbmdyZXNz IGdyb3VwIDEgcGF0dGVybiBldGggLyB2bGFuIHZpZCBpcyA3MjEgLyBlbmQgYWN0aW9ucyBv Zl9zZXRfdmxhbl92aWQgdmxhbl92aWQgNzIyIC8gcnNzIHF1ZXVlcyAxIDIgMyA0IDUgNiA3 IDggZW5kIC8gZW5kDQo+PiBmbG93IGNyZWF0ZSAxIGluZ3Jlc3MgZ3JvdXAgMSBwYXR0ZXJu IGV0aCAvIHZsYW4gdmlkIGlzIDcyMSAvIGVuZCBhY3Rpb25zIG9mX3NldF92bGFuX3ZpZCB2 bGFuX3ZpZCA3MjIgLyByc3MgcXVldWVzIDEgMiAzIDQgNSA2IDcgOCBlbmQgLyBlbmQNCj4+ IGZsb3cgY3JlYXRlIDAgaW5ncmVzcyBncm91cCAxIHBhdHRlcm4gZXRoIC8gdmxhbiB2aWQg aXMgNzIyIC8gZW5kIGFjdGlvbnMgb2Zfc2V0X3ZsYW5fdmlkIHZsYW5fdmlkIDcyMSAvIHJz cyBxdWV1ZXMgMSAyIDMgNCA1IDYgNyA4IGVuZCAvIGVuZA0KPj4gZmxvdyBjcmVhdGUgMSBp bmdyZXNzIGdyb3VwIDEgcGF0dGVybiBldGggLyB2bGFuIHZpZCBpcyA3MjIgLyBlbmQgYWN0 aW9ucyBvZl9zZXRfdmxhbl92aWQgdmxhbl92aWQgNzIxIC8gcnNzIHF1ZXVlcyAxIDIgMyA0 IDUgNiA3IDggZW5kIC8gZW5kDQo+PiBmbG93IGNyZWF0ZSAwIGluZ3Jlc3MgZ3JvdXAgMCBw YXR0ZXJuIGVuZCBhY3Rpb25zIGp1bXAgZ3JvdXAgMSAvIGVuZA0KPj4gZmxvdyBjcmVhdGUg MSBpbmdyZXNzIGdyb3VwIDAgcGF0dGVybiBlbmQgYWN0aW9ucyBqdW1wIGdyb3VwIDEgLyBl bmQNCj4+DQo+Pj4gRm9yIHBhY2tldHMgaW4gdGhlIHJhbmdlIDY4LTgwQiBJIG1lYXN1cmVk IGV2ZW4gbG93ZXIgdGhyb3VnaHB1dC4NCj4+PiBGdWxsIHRocm91Z2hwdXQgSSBtZWFzdXJl ZCBvbmx5IGZyb20gcGFja2V0cyBsYXJnZXIgdGhhbiAxMTJCDQo+Pj4NCj4+PiBGb3Igb25s eSBvbmUgcXVldWUsIEkgZGlkbid0IGdldCBtb3JlIHRoYW4gNTVHYnBzOg0KPj4+ICQgc3Vk byBkcGRrLXRlc3RwbWQgLWwgMC0xIC1uIDQgLWEgMDAwMDoxNzowMC4wLGhwX2J1Zl9sb2df c3o9MTkgLS0NCj4+PiAtLXJ4cT0xIC0tdHhxPTEgLS1yeGQ9NDA5NiAtLXR4ZD00MDk2IC0t aGFpcnBpbnE9MSAtaQ0KPj4+IHRlc3RwbWQ+IGZsb3cgY3JlYXRlIDAgaW5ncmVzcyBwYXR0 ZXJuIGV0aCBzcmMgaXMgMDA6MTA6OTQ6MDA6MDA6MDMgLw0KPj4+IGVuZCBhY3Rpb25zIHF1 ZXVlIGluZGV4IDEgLyBlbmQNCj4+Pg0KPj4+IEkgdHJpZWQgdG8gdXNlIGxvY2tlZCBkZXZp Y2UgbWVtb3J5IGZvciBUWCBhbmQgUlggcXVldWVzLCBidXQgaXQgc2VlbXMNCj4+PiB0aGF0 IHRoaXMgaXMgbm90IHN1cHBvcnRlZDoNCj4+PiAiLS1oYWlycGluLW1vZGU9MHgwMTEwMDAi IChiaXQgMTYgLSBoYWlycGluIFRYIHF1ZXVlcyB3aWxsIHVzZSBsb2NrZWQNCj4+PiBkZXZp Y2UgbWVtb3J5LCBiaXQgMTIgLSBoYWlycGluIFJYIHF1ZXVlcyB3aWxsIHVzZSBsb2NrZWQg ZGV2aWNlIG1lbW9yeSkNCj4+IFJ4USBwaW5uZWQgaW4gZGV2aWNlIG1lbW9yeSByZXF1aXJl cyBmaXJtd2FyZSBjb25maWd1cmF0aW9uIFsxXToNCj4+DQo+PiBtbHhjb25maWcgLXkgLWQg JHBjaV9hZGRyIHNldCBNRU1JQ19TSVpFX0xJTUlUPTAgSEFJUlBJTl9EQVRBX0JVRkZFUl9M T0NLPTENCj4+IG1seGZ3cmVzZXQgLXkgLWQgJHBjaV9hZGRyIHJlc2V0DQo+Pg0KPj4gWzFd Omh0dHBzOi8vZG9jLmRwZGsub3JnL2d1aWRlcy9wbGF0Zm9ybS9tbHg1Lmh0bWw/aGlnaGxp Z2h0PWhhaXJwaW5fZGF0YV9idWZmZXJfbG9jaw0KPj4NCj4+IEhvd2V2ZXIsIHBpbm5lZCBS eFEgZGlkbid0IGltcHJvdmUgYW55dGhpbmcgZm9yIG1lLg0KPj4NCj4+IFR4USBwaW5uZWQg aW4gZGV2aWNlIG1lbW9yeSBpcyBub3Qgc3VwcG9ydGVkIGJ5IG5ldC9tbHg1Lg0KPj4gVHhR IHBpbm5lZCB0byBEUERLIG1lbW9yeSBtYWRlIHBlcmZvcm1hbmNlIGF3ZnVsIChwcmVkaWN0 YWJseSkuDQo+Pg0KPj4+IEkgd2FzIGV4cGVjdGluZyB0aGF0IGFjaGlldmluZyBmdWxsIHRo cm91Z2hwdXQgd2l0aCBoYWlycGluIHF1ZXVlcyB3b3VsZA0KPj4+IG5vdCBiZSBhIHByb2Js ZW0uDQo+Pj4gSXMgbXkgZXhwZWN0YXRpb24gdG9vIG9wdGltaXN0aWM/DQo+Pj4NCj4+PiBX aGF0IG90aGVyIHBhcmFtZXRlcnMgYmVzaWRlcyAnaHBfYnVmX2xvZ19zeicgY2FuIEkgdXNl IHRvIGFjaGlldmUgZnVsbA0KPj4+IHRocm91Z2hwdXQ/DQo+PiBJbiBteSBleHBlcmltZW50 cywgZGVmYXVsdCAiaHBfYnVmX2xvZ19zeiIgb2YgMTYgaXMgb3B0aW1hbC4NCj4+IFRoZSBt b3N0IGluZmx1ZW50aWFsIHBhcmFtZXRlciBhcHBlYXJzIHRvIGJlIHRoZSBudW1iZXIgb2Yg aGFpcnBpbiBxdWV1ZXMuDQo+Pg0KPj4+IEkgdHJpZWQgY29tYmluaW5nIHRoZSBmb2xsb3dp bmcgcGFyYW1ldGVyczogbXBycV9lbj0sIHJ4cXNfbWluX21wcnE9LA0KPj4+IG1wcnFfbG9n X3N0cmlkZV9udW09LCB0eHFfaW5saW5lX21wdz0sIHJ4cV9wa3RfcGFkX2VuPSwNCj4+PiBi dXQgd2l0aCBubyBwb3NpdGl2ZSBpbXBhY3Qgb24gdGhyb3VnaHB1dC4NCj4NCg0K --------------6Tw3mqV5enY9qt2YU7l4oIv6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello

I did experiments where I sent packets to the hairpin queues and the CPU queue at the same time.
During testing, I found that when the CPU queue is overloaded too much, the hairpin queues also begin to drop packets.

Example 1:
Sending 10 Gbps to hairpin queues
Resulting throughput is 10 Gbps
Expected result

Example 2:
Sending 20 Gbps to CPU queue
Resulting throughput is 11 Gbps (9 Gbps drop)
Expected result

Example 3:
Sending 10 Gbps to hairpin queues and 20 Gbps to CPU queue
Resulting throughput is 21Gbps, 10 Gbps (zero packet drop) from hairpin + 11 Gbps from CPU
Expected result

Example 4:
Sending 10 Gbps to hairpin queues and 50 Gbps to CPU queue
Resulting throughput is 16 Gbps, 5Gbps (50%+ packet drop) from hairpin + 11Gbps from CPU,
Unexpected result...

Experiments setup:
sudo mlxconfig -y -d 0000:c4:00.0 set MEMIC_SIZE_LIMIT=3D0 HAIRPIN_DATA_BUFFER_LOCK=3D1
sudo mlxfwreset -y -d 0000:c4:00.0 reset
sudo dpdk-testpmd -l 0-1 -n 4 -a 0000:c4:00.0,hp_buf_log_sz=3D13 --= --rxq=3D1 --txq=3D1 --hairpinq=3D12 --hairpin-mode=3D0x1110 -i
=C2=A0=C2=A0=C2=A0 flow create 0 ingress pattern eth src is 00:10:9= 4:00:00:02 / end actions queue index 0 / end
=C2=A0=C2=A0=C2=A0 flow create 0 ingress pattern eth src is 00:10:9= 4:00:00:03 / end actions rss queues 1 2 3 4 5 6 7 8=C2=A0 end / end

So I can't achieve my goal: traffic from the hairpin queues is not dropped if the CPU queue is overloaded.
Any idea how to achieve this in example 4?
What is the problem, full packet buffers/memory in the device that are shared between the hairpin and CPU queues?

Any guidance or suggestions on how to= achieve this would be greatly appreciated.

M=C3=A1rio
On 27/06/2024 13:42, M=C3=A1rio Kuka w= rote:
Hi Dmitry,

Thank you for your helpful reply.
 
Try enabling "=
Explicit Tx rule" mode if possible.
I was able to achieve 137 Mpps @ 64B with the following command:

dpdk-testpmd -a 21:00.0 -a c1:00.0 --in-memory -- \
    -i --rxq=3D1 --txq=3D1 --hairpinq=3D8 --hairpin-mode=3D0x10

Based o this I was able to achieve 142 Mpps(96.08 Gbps) @ 64B with the fo=
llowing command:

sudo dpdk-testpmd -l 0-1 -n 4 -a 0000:c4:00.0,hp_buf_log_sz=3D13 \
    --in-memory -- --rxq=3D1 --txq=3D1 --hairpinq=3D12 --hairpin-mode=3D0=
x10 -i
   =20
flow create 0 ingress pattern eth src is 00:10:94:00:00:02 / end actions =
rss queues 1 2 3 4 5 6 7 8 9 10 11 12 end / end

Almost full speed :).
Any other value of "hp_buf_log_sz" or more queues does not get better res=
ults, but instead makes them worse.


RxQ pinn=
ed in device memory requires firmware configuration [1]:

mlxconfig -y -d $pci_addr set MEMIC_SIZE_LIMIT=3D0 HAIRPIN_DATA_BUFFER_LO=
CK=3D1
mlxfwreset -y -d $pci_addr reset

[1]: https://doc.dpdk.org/guides/platform/mlx5.=
html?highlight=3Dhairpin_data_buffer_lock

However, pinned RxQ didn't improve anything for me.

I tried it, but it didn't improve anything for me either.

M=C3=A1rio
On 25/06/2024 02:22,=C2=A0 Kozlyuk w= rote:
Hi M=C3=A1rio,

2024-06-19 08:45 (UTC+0200), M=C3=A1rio Kuka:

Hello,

I want to use hairpin queues to forward high priority traffic (such as=20
LACP).
My goal is to ensure that this traffic is not dropped in case the=20
software pipeline is overwhelmed.
But during testing with dpdk-testpmd I can't achieve full throughput for =

hairpin queues.

For maintainers: I'd like =
to express interest in this use case too.


The best result I have b=
een able to achieve for 64B packets is 83 Gbps=20
in this configuration:
$ sudo dpdk-testpmd -l 0-1 -n 4 -a 0000:17:00.0,hp_buf_log_sz=3D19 --=20
--rxq=3D1 --txq=3D1 --rxd=3D4096 --txd=3D4096 --hairpinq=3D2
testpmd> flow create 0 ingress pattern eth src is 00:10:94:00:00:03 / =
 =20
end actions rss queues 1 2 end / end

Try enabling "Explicit Tx =
rule" mode if possible.
I was able to achieve 137 Mpps @ 64B with the following command:

dpdk-testpmd -a 21:00.0 -a c1:00.0 --in-memory -- \
    -i --rxq=3D1 --txq=3D1 --hairpinq=3D8 --hairpin-mode=3D0x10

You might get even better speed, because my flow rules were more complica=
ted
(RTE Flow based "router on-a-stick"):

flow create 0 ingress group 1 pattern eth / vlan vid is 721 / end actions=
 of_set_vlan_vid vlan_vid 722 / rss queues 1 2 3 4 5 6 7 8 end / end
flow create 1 ingress group 1 pattern eth / vlan vid is 721 / end actions=
 of_set_vlan_vid vlan_vid 722 / rss queues 1 2 3 4 5 6 7 8 end / end
flow create 0 ingress group 1 pattern eth / vlan vid is 722 / end actions=
 of_set_vlan_vid vlan_vid 721 / rss queues 1 2 3 4 5 6 7 8 end / end
flow create 1 ingress group 1 pattern eth / vlan vid is 722 / end actions=
 of_set_vlan_vid vlan_vid 721 / rss queues 1 2 3 4 5 6 7 8 end / end
flow create 0 ingress group 0 pattern end actions jump group 1 / end
flow create 1 ingress group 0 pattern end actions jump group 1 / end


For packets in the range=
 68-80B I measured even lower throughput.
Full throughput I measured only from packets larger than 112B

For only one queue, I didn't get more than 55Gbps:
$ sudo dpdk-testpmd -l 0-1 -n 4 -a 0000:17:00.0,hp_buf_log_sz=3D19 --=20
--rxq=3D1 --txq=3D1 --rxd=3D4096 --txd=3D4096 --hairpinq=3D1 -i
testpmd> flow create 0 ingress pattern eth src is 00:10:94:00:00:03 / =
 =20
end actions queue index 1 / end

I tried to use locked device memory for TX and RX queues, but it seems=20
that this is not supported:
"--hairpin-mode=3D0x011000" (bit 16 - hairpin TX queues will use locked=20
device memory, bit 12 - hairpin RX queues will use locked device memory)

RxQ pinned in device memor=
y requires firmware configuration [1]:

mlxconfig -y -d $pci_addr set MEMIC_SIZE_LIMIT=3D0 HAIRPIN_DATA_BUFFER_LO=
CK=3D1
mlxfwreset -y -d $pci_addr reset

[1]: https://doc.dpdk.org/guides/platform/mlx=
5.html?highlight=3Dhairpin_data_buffer_lock

However, pinned RxQ didn't improve anything for me.

TxQ pinned in device memory is not supported by net/mlx5.
TxQ pinned to DPDK memory made performance awful (predictably).


I was expecting that ach=
ieving full throughput with hairpin queues would=20
not be a problem.
Is my expectation too optimistic?

What other parameters besides 'hp_buf_log_sz' can I use to achieve full=20
throughput?

In my experiments, default=
 "hp_buf_log_sz" of 16 is optimal.
The most influential parameter appears to be the number of hairpin queues=
=2E


I tried combining the fo=
llowing parameters: mprq_en=3D, rxqs_min_mprq=3D,=20
mprq_log_stride_num=3D, txq_inline_mpw=3D, rxq_pkt_pad_en=3D,
but with no positive impact on throughput.


--------------6Tw3mqV5enY9qt2YU7l4oIv6-- --------------ms040406050304030204000308 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC DVswggZtMIIEVaADAgECAhEAnbnQ7qfoWj/NNwoRkcscADANBgkqhkiG9w0BAQwFADBGMQsw CQYDVQQGEwJOTDEZMBcGA1UEChMQR0VBTlQgVmVyZW5pZ2luZzEcMBoGA1UEAxMTR0VBTlQg UGVyc29uYWwgQ0EgNDAeFw0yNDA0MjIwMDAwMDBaFw0yNjA0MjIyMzU5NTlaMIHXMQswCQYD VQQGEwJDWjEeMBwGA1UECAwVUHJhaGEsIEhsYXZuw60gbcSbc3RvMTgwNgYDVQQKDC9DRVNO RVQsIHrDoWptb3bDqSBzZHJ1xb5lbsOtIHByw6F2bmlja8O9Y2ggb3NvYjEZMBcGA1UEYRMQ TlRSQ1otNjM4IDM5IDE3MjEdMBsGCSqGSIb3DQEJARYOa3VrYUBjZXNuZXQuY3oxDTALBgNV BAQTBEt1a2ExDzANBgNVBCoMBk3DoXJpbzEUMBIGA1UEAwwLTcOhcmlvIEt1a2EwggEiMA0G CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCiHwBI7t3hatQXKJ1X1fANpo1f5rur//PMoRMb w06yUiWU8lPAQTGshceMnNDTy+PsZ+jvJ6H5xsrISX911zLBH2T2BcF3IVEEiu6k3JBbXMlc lJvrkKfBcOsV2rnYQUqLHxejKVaeTmvMjTwWGm13HfERf3TRL5JvE9FzwsYHY+XokiFBrHgb pN18YyjvkvS+LJcMYAPFo+jgEsu7WZY4xjK13fIzKk+qoneD8rrX5bbnhuhAfaHOpnHvowJz sQR2lLK/8A2pdLAJd4YQi8ITylZeTaIPWK6M13tc2TU2/mCXXyni4LI0q2+n2gzu4oqILCHE OVam0bRFVAxzZnmFAgMBAAGjggHCMIIBvjAfBgNVHSMEGDAWgBRpAKHHIVj44MUbILAK3adR vxPZ5DAdBgNVHQ4EFgQUqx/ddGZmQk2PwBpPufqOYZQavXUwDgYDVR0PAQH/BAQDAgWgMAwG A1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMFAGA1UdIARJMEcw OgYMKwYBBAGyMQECAQoEMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8vc2VjdGlnby5jb20vU01J TUVDUFMwCQYHZ4EMAQUDAjBCBgNVHR8EOzA5MDegNaAzhjFodHRwOi8vR0VBTlQuY3JsLnNl Y3RpZ28uY29tL0dFQU5UUGVyc29uYWxDQTQuY3JsMHgGCCsGAQUFBwEBBGwwajA9BggrBgEF BQcwAoYxaHR0cDovL0dFQU5ULmNydC5zZWN0aWdvLmNvbS9HRUFOVFBlcnNvbmFsQ0E0LmNy dDApBggrBgEFBQcwAYYdaHR0cDovL0dFQU5ULm9jc3Auc2VjdGlnby5jb20wLwYDVR0RBCgw JoEUTWFyaW8uS3VrYUBjZXNuZXQuY3qBDmt1a2FAY2VzbmV0LmN6MA0GCSqGSIb3DQEBDAUA A4ICAQAlHGBXnHAkPcBj0dtzq+AsVAHCbDjAyrvh6DjaQO6KGfsCdJiI3MKRE7Kmpm0rbdb9 xoSBjtZY5uSJO0ZkAQzz21jcTgiRUJmNgdffTk1r2JaFpXpbQSKrtjOfVG6HfUvOfbgLyVev A7HH837SaniLCaqj2OzBPH2slV+Q+lYxU2W6pP/S/4tjHvypLruKDsp7GFLwDUYbKm7/vBSK prBMnjkLq2HQmQ+E3LkPEt8L5In8BpFpKRmAFj+cNm14xv715o+JreioBG1JklsWkFY2X7KE KYRM6Ov2oJuqK2plhviM4c3r8yQ5bTvale/UWPcm6GXOrGRJXaR0yjuHWk0MZcwrHqXQbfiO porDmt68zVK05UacMRB0RrEQ+1nEjzzsHTVXzpHIIbCTzKBiEadHsaka3mWMFrLP2fWDh1m4 IRTsKJy9YwYmjzDKYMAdisAXgXE4QHN2K8LLHLMttBGoOI08X91pfSf5WoB4g8UM5+tuemSo uNpj14gHxFrxoGCC53EImN0f9F/Lh116CpJKDLN+06RYyeVKNjUVWONZfCeEcQLGWNJTDh3p IDLl13COJGbtUwr2cjHTi1WFG4tYYJUadGgih5+ObDgk9CKqXFFLvkQIXOp7qQnRgRd+Ochq nCGaOZhLHsd/O7ByimWJ3OB6nDujOahK4PRaHS+G8zCCBuYwggTOoAMCAQICEDECcNQ1vpsk mvhW0OHihUkwDQYJKoZIhvcNAQEMBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpOZXcg SmVyc2V5MRQwEgYDVQQHEwtKZXJzZXkgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBO ZXR3b3JrMS4wLAYDVQQDEyVVU0VSVHJ1c3QgUlNBIENlcnRpZmljYXRpb24gQXV0aG9yaXR5 MB4XDTIwMDIxODAwMDAwMFoXDTMzMDUwMTIzNTk1OVowRjELMAkGA1UEBhMCTkwxGTAXBgNV BAoTEEdFQU5UIFZlcmVuaWdpbmcxHDAaBgNVBAMTE0dFQU5UIFBlcnNvbmFsIENBIDQwggIi MA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCzSuIiXidb6QRbFAQ1MiAUrrTSMUDGzVDA HqFEyq+eSmF/LZDeYpszai2kQsqWATz/cBA9gGjunvJ45G48ycC4D6gwZFvbBt5JotxlunBe B8K+crGar3v+RCQ4VfvToX07v+HTJ6EeEONR3IzJPTMyzgAwENGsAWf9va9HePQFJiChCzXq hKpp0zen53S+8f9itEy06GS8aku7Mvyb4tMBBa9An3Y3ALIqIeymg/iYs8m9WkSkMyekNtFR B1+1KlnNUpM05G8+sY9EucnQQRUIdHzYsvqjP3XlaSuB4Jj0ia66UGfi5Wx31mm5sKAz8Re9 UGVWIqq6wKFKxkSfuO4iwYiIPJoiGEux3dqabwFLduAroDF1IxE40PqGIdPXzYuZ/wL6BEfF Ab0xy8bfm5S9G7y/ts9mIlFpPtkLZ/nQ/iVOWdsu9ale/nK/uGF47xsxeW2LIvB4sH5U2+D4 ad8vpNbcCrXIXXKtkBnNHgxumNNZ0R1Isq/Pz7TALCxxDzWdsM7AO32/Jn7R4ldtGRZmKpJy fACDn8HU1QPhOtiWsjifrMWnanJhQ6K7M/5qz8BmfPrca+MUrr1Y4NHZb9MrgPtWKQQyGDXy +G5F/iHGdZk7LS+F8NH+Ddolt2wZpz52JqGMTDPIH5Qok4LLO95wbxtn+79Tw+wQxmDTuIg+ LwIDAQABo4IBizCCAYcwHwYDVR0jBBgwFoAUU3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0O BBYEFGkAocchWPjgxRsgsArdp1G/E9nkMA4GA1UdDwEB/wQEAwIBhjASBgNVHRMBAf8ECDAG AQH/AgEAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDA4BgNVHSAEMTAvMC0GBFUd IAAwJTAjBggrBgEFBQcCARYXaHR0cHM6Ly9zZWN0aWdvLmNvbS9DUFMwUAYDVR0fBEkwRzBF oEOgQYY/aHR0cDovL2NybC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJTQUNlcnRpZmljYXRp b25BdXRob3JpdHkuY3JsMHYGCCsGAQUFBwEBBGowaDA/BggrBgEFBQcwAoYzaHR0cDovL2Ny dC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJTQUFkZFRydXN0Q0EuY3J0MCUGCCsGAQUFBzAB hhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0GCSqGSIb3DQEBDAUAA4ICAQAKBU57DY8f EzkA/W/sYsbD7e0XquMBzHjcP0eXXXRD4EAEAGCWSs+QRL9XIxmx+52zx9wMa8YTejlR+NKe jiyKPTF0q63zMxrO/z/hUwo8IDcRRLS0NSgvTW6AN2rCXJe5iLN5fIfYgIBB9cy1L6trPuZ/ vjUJm87nQ7ExQzGqWN5F9U8MlAk0c5iLanG7GCMoNjHiF1n0baj6guUeG7n5qcwOQTyDS19+ NEqfwjUPUGasN1ZH8h1sE6PrzvRpti+rKzWpiU+i2/k3l2b5fFDy+Wu9jv6R9BoBh47es/UM zwEZ2kSrIVVr4jSukk+FpmR5ZbtwiYNAV6sdb1srMGsILzXlrdasSE2nGHvZklk2zUdgn7b0 1MHq67g0mNozGmT6Dam41Kbhv25WMFs871XqwVIb4gGoT1yRf/VePMm1jwauqijhKJFvrNwe GnebGPeipaLxIo2iEA4qdRztEg/qyzWGogXK/TFdmivg322fMPQWjQkMhRGMM8SCjlZN22L8 x0ZOYoVA2rHJm5P25IjZe+HPyn7ikJiSJmqlqFmUeowXF3D1dFlCCs/5yC06RYRqI2REFu+2 8t2nswIvY6xCFAR0RtS8Mz2yXNld0js2MmiRUGrc7imWzdUPbPcv9sdUF7SsERGPIzYL8dIi Hzit+YCoGCSXMg6peF37hHNp1TGCAzswggM3AgEBMFswRjELMAkGA1UEBhMCTkwxGTAXBgNV BAoTEEdFQU5UIFZlcmVuaWdpbmcxHDAaBgNVBAMTE0dFQU5UIFBlcnNvbmFsIENBIDQCEQCd udDup+haP803ChGRyxwAMA0GCWCGSAFlAwQCAQUAoIIBsTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0yNDA3MDQxMTA4MTBaMC8GCSqGSIb3DQEJBDEiBCA7 +FeAHBBviI7uwvA00yTVynT5M5kK4sRjR+/bFls7SjBqBgkrBgEEAYI3EAQxXTBbMEYxCzAJ BgNVBAYTAk5MMRkwFwYDVQQKExBHRUFOVCBWZXJlbmlnaW5nMRwwGgYDVQQDExNHRUFOVCBQ ZXJzb25hbCBDQSA0AhEAnbnQ7qfoWj/NNwoRkcscADBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCG SAFlAwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqG SIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMGwGCyqGSIb3DQEJEAILMV2gWzBG MQswCQYDVQQGEwJOTDEZMBcGA1UEChMQR0VBTlQgVmVyZW5pZ2luZzEcMBoGA1UEAxMTR0VB TlQgUGVyc29uYWwgQ0EgNAIRAJ250O6n6Fo/zTcKEZHLHAAwDQYJKoZIhvcNAQEBBQAEggEA GUmDTR6mJmCqalfyRrc4t11XVkF/LyxkzjBwW7M9MgC7ttThmGm2CoRbGmCAZaamqUBV1DLS ZP1iuarD9oNl9GSXE4NCWEyl8OHmCUOWEQTp7T+W20V++SXbvNm5/bF8IQoVR3rp05p+bhuT vyHnWwSsMPNlBaF+Ak4CCmC0O1hJ8jstcNchp9SsgTvFBur0tZHr8OArdoEW8cAreChIBrIb RQJR+iylqK59e2H5eHDMhHW74IuWtwPDKgZqa214jbx/lgi4Yam8HbCsIJ2aYzdOXOzywF04 tNXoQ35a2Cq6O3nP6WsPZqKxWMbi07mbYnfhmGJ2Rrv4DYpPQFXzRAAAAAAAAA== --------------ms040406050304030204000308--