From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from BAY004-OMC4S2.hotmail.com (bay004-omc4s2.hotmail.com [65.54.190.204]) by dpdk.org (Postfix) with ESMTP id 8B852B6D for ; Sat, 28 Jan 2017 20:57:09 +0100 (CET) Received: from EUR03-AM5-obe.outbound.protection.outlook.com ([65.54.190.199]) by BAY004-OMC4S2.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Sat, 28 Jan 2017 11:57:08 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mVgNZcEX87xapWpcrXiK05CKOBIKjKBpy8yL5HRshiM=; b=kDSwhG0cbKi4UHss4ZHciB1pekxyHxj0Atp6p+Hk0EVNqbeMTal8XtMskjtjbBs5G6ohDsQimP7SNo6qyDSIRvJLBdEZOWak3CHWdO3U7xL+I2m3ELnoVEMdlflNHuZP4y7W6i5dzJGAXPBycurpttvFefeleyixxcAAoAm2d7WvllYIt1NQprnT4d9TwoE3Tj5C31OdZ3pzVnK4kxjXWPPUuU8cOaz5ioXH1/H+IVLh5rwmUwgNTJxo7sJPGpxeU9K5lDwP+2GDyWEg6AB/cOlWQG5ILKiZh6laO/QQqhJMAwS5JBeuxrGuRCf+kXsGEcvxhcZPyYOIsHEV4RDQJQ== Received: from VE1EUR03FT009.eop-EUR03.prod.protection.outlook.com (10.152.18.60) by VE1EUR03HT256.eop-EUR03.prod.protection.outlook.com (10.152.19.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8; Sat, 28 Jan 2017 19:57:04 +0000 Received: from DB6PR1001MB1416.EURPRD10.PROD.OUTLOOK.COM (10.152.18.53) by VE1EUR03FT009.mail.protection.outlook.com (10.152.18.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.874.2 via Frontend Transport; Sat, 28 Jan 2017 19:57:04 +0000 Received: from DB6PR1001MB1416.EURPRD10.PROD.OUTLOOK.COM ([10.171.78.136]) by DB6PR1001MB1416.EURPRD10.PROD.OUTLOOK.COM ([10.171.78.136]) with mapi id 15.01.0874.018; Sat, 28 Jan 2017 19:57:04 +0000 From: Peter Keereweer To: "users@dpdk.org" Thread-Topic: What to do after rte_eth_tx_burst: free or send again remaining packets? Thread-Index: AQHSeZ89QhXZzOqpSEWE4fljFDCT1aFOTees Date: Sat, 28 Jan 2017 19:57:04 +0000 Message-ID: References: In-Reply-To: Accept-Language: nl-NL, en-US Content-Language: nl-NL X-MS-Has-Attach: X-MS-TNEF-Correlator: x-slblob-mailprops: 6H6McBavlAjyIid6LH+FfLg5EkryiYUMBP6Sn9Jhvw3i3tvukGYJ6aBl9ciAU/STVZVkVMfXXwzMKOKy/GiDa5aQF87N7BUnd0+BqPJPAv3ADrMnXeF61r02LEV9NcaUjYi2nIFwHaDoSZODIdoVyQHXPo/xthrXxw1OnYAlnNPPXqAn+gcTVSnatMW/E9wqgPCXKKfS7MJpxHnBPXikvdnvU3Zv+xVi1FHjDJ9IeOU/4UzYJlckKlbVRs0a2QQ2nRsDpyCm8H3IT39BSD0hso+N/V7zJxA9rLY98soRsRfrcUhHlVo8pgYuUNOsaQTjIhkeNPDQobR4WBq6L04mwOjD/6JLvGulSxHYIpCYdlo83nrzV4MYSHEh1zbuLHunUlTgyfPwWOEIeA5R/oN7eLeoEBTjhjeKQ0m/ppqGq245B412ebnz93ETlZfONqcV5Lc1GhX5BzGJiZZQeG2lqAI12FE1Xa7N9sNaBoGjuniVSinSo+s67zByNCSnMuhemLmhuor/tZ9sAb/cnWMSw1SvyAxP5TUBE0HUcwDf27WAfcZjD1z9ATIUda5WrrVfNy2Jg6/PLq9UHQ8iZIf/BUPkYq9hyp3ZNT70HLloDhjxu98XCU/CeQ0laouZ1al8qUth8gt/hOHGI+m8fkyZ6PDivCVibcYJWn9TvnIjbvKFbjMkAQgq3CkqUXwNI2A1WOdOhuxIVxLKjsDR8IkbGgKFHZgKtAAemM9kug6s0rL+imUNy4AotaoRnc0f8O+opiascd026fw= authentication-results: hotmail.com; dkim=none (message not signed) header.d=none;hotmail.com; dmarc=none action=none header.from=hotmail.com; x-incomingtopheadermarker: OriginalChecksum:C1C6E65248A8D6739833A7E04963CDAB93C59321997939A6B2FC6115C6C8426E; UpperCasedChecksum:999B62DB5ABAF1B356EBDD635519DC5DB3A2731E68874B88F92FD51FE04677E6; SizeAsReceived:8734; Count:40 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [j2gA6KrwyqTGiideNP7kyf7A1nSc9Ql4] x-incomingheadercount: 40 x-eopattributedmessage: 0 x-microsoft-exchange-diagnostics: 1; VE1EUR03HT256; 7:mTYrrkaXM7fcxccbK50a5pEx6iGu3FBKjbSt3ywQHz+AdDmVSplaIilQp1PdMoRt7jxAJw2hoVIjEv2octiyzYHeblclVIz8oA05aWdjiF4wwTtao4fxCpMHvpVFljxLDzuFKs1/R8dgam8tZk31lHFQMnoWNP/fgQ5YxUmUDMsw5xPOfdWW2Zx1SqH41FWQM9f6WGVGKlompkVftQYjOnEGD0TVt0ntbUFzIfmO5HTw/TT8IaUfwWKKH06uITAoBYzf5yNWtXGGF6ln6oVWYJO9TVnaKXNIp/IyTU4mo4ruVFarOEa1Wu8lVW0DQduCciAJYoqr571lsCMcXZAeNlAwA7/Clt1I0txdkmOjqormQLDrLMvHsFDBuw1+a+vPNUy/V50vcEEEe3mPpVNTFD1U1tSpqPGdY/MMPdaVNPuBq9kiUoBfEszHoL1uK9b312GUm4Q9gp3ttEFjIonFnQ== x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900005); DIR:OUT; SFP:1102; SCL:1; SRVR:VE1EUR03HT256; H:DB6PR1001MB1416.EURPRD10.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: 3990ee14-f416-4fb1-5ca6-08d447b7d512 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(1601124038)(5061506344)(1603103113)(1601125047)(1701031023); SRVR:VE1EUR03HT256; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444111334)(444112120)(432015012)(82015046); SRVR:VE1EUR03HT256; BCL:0; PCL:0; RULEID:; SRVR:VE1EUR03HT256; x-forefront-prvs: 02015246A9 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jan 2017 19:57:04.4159 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1EUR03HT256 X-OriginalArrivalTime: 28 Jan 2017 19:57:08.0529 (UTC) FILETIME=[B4FBBA10:01D279A0] Subject: [dpdk-users] What to do after rte_eth_tx_burst: free or send again remaining packets? X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2017 19:57:10 -0000 Hi! Currently I'am running some tests with the Load Balancer Sample Application= . I'm testing the Load Balancer Sample Application by sending packets with = pktgen. I have a setup of 2 servers with each server containing a Intel 10Gbe 82599= NIC (connected to each other). I have configured the Load Balancer applica= tion to use 1 core for RX, 1 worker core and 1 TX core. The TX core sends a= ll packets back to the pktgen application. With the pktgen I send 1024 UDP packets to the Load Balancer. Every packet = processed by the worker core will be printed to the screen (I added this co= de by myself). If I send 1024 UDP packets, 1008 ( =3D 7 x 144) packets will= be printed to the screen. This is correct, because the RX core reads pack= ets with a burst size of 144. So if I send 1024 packets, I expect 1008 pack= ets back in the pktgen application. But surprisingly I only receive 224 pac= kets instead of 1008 packets. After some research I found that that 224 pa= ckets is not just a random number, its 7 x 32 (=3D 224). So if the RX reads= 7 x 144 packets, I get back 7 x 32 packets. After digging into the code fr= om the Load Balancer application I found in 'runtime.c' in the 'app_lcore_i= o_tx' function this code : n_pkts =3D rte_eth_tx_burst( =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 port, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 0, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 lp->tx.mbuf_out[port].array, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 (uint16_t) n_mbufs); ... if (unlikely(n_pkts < n_mbufs)) { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 uint32_t k; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 for (k =3D n_pkts; k < n_mbufs; k ++) { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 struct rte_mbuf *pkt_to_free =3D= lp->tx.mbuf_out[port].array[k]; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 rte_pktmbuf_free(pkt_to_free); =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 } =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } What I understand from this code is that n_mbufs 'packets' are send with 'r= te_eth_tx_burst' function. This function returns n_pkts, the number of pack= ets that are actually send. If the actual number of packets send is smaller= then n_mbufs (packets ready for send given to the rte_eth_tx_burst) then = all remaining packets, which are not send, are freed. In de the Load Balanc= er application, n_mbufs is equal to 144. But in my case 'rte_eth_tx_burst' = returns the value 32, and not 144. So 32 packets are actually send and the= remaining packets (144 - 32 =3D 112) are freed. This is the reason why I g= et 224 (7 x 32) packets back instead of 1008 (=3D 7 x 144). But the question is: why are the remaining packets freed instead of trying = to send them again? If I look into the 'pktgen.c', there is a function '_se= nd_burst_fast' where all remaining packets are trying to be send again (in = a while loop until they are all send) instead of freeing them (see code be= low) : static __inline__ void _send_burst_fast(port_info_t *info, uint16_t qid) { =A0=A0=A0=A0=A0=A0=A0 struct mbuf_table=A0=A0 *mtab =3D &info->q[qid].tx_mb= ufs; =A0=A0=A0=A0=A0=A0=A0 struct rte_mbuf **pkts; =A0=A0=A0=A0=A0=A0=A0 uint32_t ret, cnt; =A0=A0=A0=A0=A0=A0=A0 cnt =3D mtab->len; =A0=A0=A0=A0=A0=A0=A0 mtab->len =3D 0; =A0=A0=A0=A0=A0=A0=A0 pkts=A0=A0=A0 =3D mtab->m_table; =A0=A0=A0=A0=A0=A0=A0 if (rte_atomic32_read(&info->port_flags) & PROCESS_TX= _TAP_PKTS) { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 while (cnt > 0) { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ret = =3D rte_eth_tx_burst(info->pid, qid, pkts, cnt); =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 pktge= n_do_tx_tap(info, pkts, ret); =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 pkts = +=3D ret; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cnt -= =3D ret; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } =A0=A0=A0=A0=A0=A0=A0 } else { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 while(cnt > 0) { =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ret = =3D rte_eth_tx_burst(info->pid, qid, pkts, cnt); =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 pkts = +=3D ret; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cnt -= =3D ret; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } =A0=A0=A0=A0=A0=A0=A0 } }=20 Why is this while loop (sending packets until they have all been send) not = implemented in the 'app_lcore_io_tx' function in the Load Balancer applicat= ion? That would make sense right? It looks like that the Load Balancer appl= ication makes an assumption that if not all packets have been send, the re= maining packets failed during the sending proces and should be freed. I hope someone can help me with this questions. Thank you in advance!! Peter=