From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB7FEA0C47 for ; Wed, 14 Jul 2021 19:05:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 683654068A; Wed, 14 Jul 2021 19:05:17 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 58F8A4014D for ; Wed, 14 Jul 2021 19:05:15 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10045"; a="190073854" X-IronPort-AV: E=Sophos;i="5.84,239,1620716400"; d="scan'208";a="190073854" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2021 10:05:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,239,1620716400"; d="scan'208";a="562503001" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga004.jf.intel.com with ESMTP; 14 Jul 2021 10:05:14 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 14 Jul 2021 10:05:13 -0700 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 14 Jul 2021 10:05:13 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10 via Frontend Transport; Wed, 14 Jul 2021 10:05:13 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.174) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.10; Wed, 14 Jul 2021 10:05:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TEJU5FMqnPj0sBxhC/XHj1+v5oeLsGseyCtTs5harGXwwQcnfkORnDWFHvANxZXMKkTd5Ve9FEcEp+8VpIfY8urQxdGADWrK3cep2FBE/ho3Ge6ZCF2yPWBKp0vZ8p0B4EZPSUEZU37SKStDgRVj3p8a7/jzaEfM12H6f1dfNLd5HS4yIqdv8cuRN6P0As7SjAZHnxNNxW7UFzxKPzMOOnHmN6nUXXcrmjOYPYIT1kPv8u/Icaed0MNZLmZ11SxMuG1JVbKK6bA3ZcSLvgzXYi8L4dlLYooSkJei1vqgOHZ0LQGd5TQv7adxYNqUsq3sjow7Rlh7/Yx5Pytr9P6dfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I1bjD+2xj2F/4gbTu95ogiqZt8g1Y8U7F1HnfV9HLVU=; b=Z63X4ihY6cJT8wlGetgftS+uuLGqHZEr+zhVXeiIiWNiHJkZSGR5MgVKk25DK+fBanUTqj/ZECyJUgwKgWgupn20rGC059l3r7BgyVWfEHyVAGQwYB8gy9u956LUAMCirGCyRWro5N9GwNLe4NUctBOMHqRUhG6Utffus6Q6KFgsEDpdEuN+bU990wZn71fvXHh8rT7RJ4OE4Ssfvoo1XIiWoo5vzMHQi+hy8mvx/fDAtXwwb1pVJHDlVmA+LxCjKDV/lY6FHatBJVdCk9CmwDeXXPM3vuo6nlYWR82a0SfhJYrJ1KA+EpQMHKc+mDIL0Q6+50l4MIs2faKEJ9xCqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I1bjD+2xj2F/4gbTu95ogiqZt8g1Y8U7F1HnfV9HLVU=; b=mZZnV4RR092Z8JwuoasS33paPD9SegTCVcjKaHI/q8TLTYCPS7VbwfWrO4ATPiHGSH/awNyUv7Xwc7aHHuas20A+fOcayEDwQSqOh92RPuS5+0d/BSnTxlzBeyfQ0NJhQd4Fe+KT/KFY/FMfDXPxe3Utfv/mA6Om3vdPgF+hbKs= Received: from DM8PR11MB5702.namprd11.prod.outlook.com (2603:10b6:8:21::10) by DM5PR11MB1852.namprd11.prod.outlook.com (2603:10b6:3:110::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Wed, 14 Jul 2021 17:05:12 +0000 Received: from DM8PR11MB5702.namprd11.prod.outlook.com ([fe80::80a9:b7f:4717:d21d]) by DM8PR11MB5702.namprd11.prod.outlook.com ([fe80::80a9:b7f:4717:d21d%6]) with mapi id 15.20.4308.027; Wed, 14 Jul 2021 17:05:11 +0000 From: "Van Haaren, Harry" To: =?iso-8859-1?Q?G=E1bor_LENCSE?= , "users@dpdk.org" CC: "raja.nirmalraj@gmail.com" , Stephen Hemminger Thread-Topic: [dpdk-users] Dpdk multi core packet acquisition. Thread-Index: AQHXeIrylwNt5ljxGUeGZ+NIyUWo/qtClRQAgAAVaICAAAXPAA== Date: Wed, 14 Jul 2021 17:05:11 +0000 Message-ID: References: <20210714081530.07d2cbbd@hermes.local> <4b5c74ba-bca4-a0eb-680f-761207162a1f@hit.bme.hu> In-Reply-To: <4b5c74ba-bca4-a0eb-680f-761207162a1f@hit.bme.hu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: hit.bme.hu; dkim=none (message not signed) header.d=none;hit.bme.hu; dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: aaba821a-5acb-4401-f4e3-08d946e98ad1 x-ms-traffictypediagnostic: DM5PR11MB1852: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YYxBpm+lhXVKyUOMU8dT6G+kinK/oVYFCa4jjAYvyk3IYo780ATb72hHW8lqEJSGvnTisU7rUtg4Ui0bMjiIM/TSaKgFqGwhejVhDIV47CGXjmwx4NGR9a0vh2rUmRmJgZkBFsV6o5wgupM5YzxWnmCl6vzQpm7pslkwuNIwcajZ+ZRu5T583LrJuBrnVBwKK+844+4tgUUS9CKdawv65EL6nna67N/5pkYLgxwiXvgVkz/vtu4m1D5tDXcn3pDBx4RqXneyy9Rvd5Qb2BHaNW98aeP0oHT4Jp0K4gFOPSxxpZ5+D1Hg7wO/oDHUDPQZRRCaQ4NMQEED3ukMiF9n1SNcv1JCX06k5kqbc0L4xafrpfnYh446uuBtx5uLXR8AXP5bE7CnEBfEi3ccooF3DBMvccxsBWY02lRSSHqBQ4wZl0w1n0mxPaG2Q1Yc3cQ5Y6Uf4qSFcBL+aVwxEX3w/rLU9vvA4Eudn1zZNLvKY7arkhXtKGYMg6v3TvWvcNUO/7ZWmBtvkc+ik2mh9jFBNtLn0DB2SdeK0MzwopSlFEbAO3fOSicXtYuawD/uQlRbfOmkCVJ/egeiDaYPBgtnvoFNuvkge418hXUp+5Yt18h9Az2zBy56g9YFd07coxNgP2TLcOXCzyYvtel+JxcsMh2Llm97iONhTJk7eSJqcdN1lmavfGyA70fyospOWsCV36KefdETy2J1p34pEc8vNf2NkWMq2t0HBRuj1EeLh1IGvIIXGyCSTGgXkF6Zm+Z5C+dN8j5ve0LjWwF96fQqQpu6kJsp1V4rEbguP29Q2KE= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM8PR11MB5702.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39850400004)(376002)(366004)(136003)(346002)(110136005)(38100700002)(8936002)(33656002)(7696005)(71200400001)(66476007)(966005)(54906003)(52536014)(122000001)(66556008)(53546011)(316002)(6506007)(9686003)(55016002)(66574015)(66446008)(64756008)(4326008)(8676002)(478600001)(26005)(2906002)(76116006)(5660300002)(66946007)(83380400001)(186003)(86362001)(38070700004); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?PFvQMU+jMVEtH1rKKonUN/1s6PDOafILV3ZspXIauyOioV4D0B9DJ8f9eV?= =?iso-8859-1?Q?24Qoti5hw9a5KXu5eajKLU4IgyTK+e6+q67Zl1Q6X9y+xjUGL6RHfyRbJm?= =?iso-8859-1?Q?nQmuDZJlS4jiEWO+hhfDBhVrhH7n6QoEA7CK7wrUA7gKQvNCp4ENpHUe+G?= =?iso-8859-1?Q?p4kZmGmFU4QNC+umAusVONltRYWJ+Le0Kv+qg6vE/3rSveJWaZdHASoNlq?= =?iso-8859-1?Q?lEz61EoWF0NE7SC/EZKBpOx77UajaL/dqgJKRmTFL8vzyLdUx9WDgWAAEb?= =?iso-8859-1?Q?UGaTfm4KuiNw6NPTMpPmca3LIwsNJILYcLI7cgvgSghdvjzEPpIUcZ9TPP?= =?iso-8859-1?Q?tae/tbapAzqv7Q/JeYrhO5VrWlczaR8/HhPd97xteNipDrPPbAnSjDdFF0?= =?iso-8859-1?Q?lXPj98nRtabF2GqtiAdckCcWzYkz06oRmkByDbV70Tdu0gJXNemLKFBj0h?= =?iso-8859-1?Q?y+vmL5wEF+iHpiwbw9nafZzkSAHg90xl7TR1lpidAFKJM118eXs+zg0WfU?= =?iso-8859-1?Q?ub3b2GtFm8PF6qgtOCoUky6H8oacqC4vQcyki9IFUkZjwTlB3TUWKG+5oG?= =?iso-8859-1?Q?BCX0R1gGGrv93eXGZxKNAEMH0qgLz2pF2Vc0l8dtczWSU48kY5utgvqZ67?= =?iso-8859-1?Q?XZOx3YC1TX9q9j6ce+TOKSGL0I2HljufN6epIrtKIjwjWgKM3tH/tF8BAJ?= =?iso-8859-1?Q?dm/2JDpNuoC6+2uBKT3n6Rl1YutUKMz7WPVTWKr3pmHoAVi8VJNBYq7/Nc?= =?iso-8859-1?Q?2R89YgCtJhd3WTYTQW3g/CmCP/qqgFKN5p7Ewz6lbcvfFybv9NvMy/eaVo?= =?iso-8859-1?Q?fyXc+QEGnimDSuCFEKVf+MnoD13fnQsQLGE5wSG5l6lC0OJQnYdyfTeb+p?= =?iso-8859-1?Q?VbkRq4dnNovjK9ki6Es4QEbU+Pl38OW2hlQJPtm4wyf2bgBWinFZ43NObl?= =?iso-8859-1?Q?2gAP8cxTUF5aVeRTdEWYhKuDEs5rcKCDcau0EVYHyeK3EH1WZMPaFy9wNX?= =?iso-8859-1?Q?6Anss6L8tP6CcXmWdT9+1ZrjlgaydqVgRk+dYodB+yhxX5bbo46vI+9n/L?= =?iso-8859-1?Q?MltNiQQxl5fMELhkwqOsSkErkECX9Aw4DV2RS2BLo3jncUMKQ2AAOHyB1M?= =?iso-8859-1?Q?EZRQ25mhAiNlPYBBTQULZD0n/RsazPxhVg+kF8cCZv6yvVw5uPhDD6EaLf?= =?iso-8859-1?Q?xpoRIacVQEr5LTlXZYUsfDp+iNOyihvo3qk0nGKkP53lvMtXcbggs8o+YL?= =?iso-8859-1?Q?WQcqDL+a2IlKZcft4aDQY2Kf0v1Bw85HSMdvhLOR+2DnCauK+6u9/6bm5F?= =?iso-8859-1?Q?pX2TvFzFcw+q6WVj8msDCK68jryw4zQISVtfdaxu/6vi6tU=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8PR11MB5702.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: aaba821a-5acb-4401-f4e3-08d946e98ad1 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jul 2021 17:05:11.8492 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sAV3BqOENhDYbde+/jmz+sJu94bcXy9mZAMuCjYFYw20gpQ0NQMrahS0i39usM8kQcBSGAHLVtppw2x9S1zpkD6dBILg4xOEldyGIoenALw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR11MB1852 X-OriginatorOrg: intel.com Subject: Re: [dpdk-users] Dpdk multi core packet acquisition. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" (+CC Raja and Stephen) > -----Original Message----- > From: users On Behalf Of G=E1bor LENCSE > Sent: Wednesday, July 14, 2021 5:32 PM > To: users@dpdk.org > Subject: Re: [dpdk-users] Dpdk multi core packet acquisition. >=20 > I recommend you to use multi queue receiving (RSS). Thus you do not need > to use mutual exclusion. (Performance!) > Of course, you need some entropy for the hash function to distribute the > packets approximately evenly among the queues. >=20 > G=E1bor Good input above! Just for future reference, please don't top-post on mailing lists, and always use the "reply all" to keep everybody on CC. Top posting detail: https://www.mediawiki.org/wiki/Mailing_list_etiquette > 2021.07.14. 17:15 keltez=E9ssel, Stephen Hemminger =EDrta: > > On Wed, 14 Jul 2021 10:32:56 +0200 > > Nirmal R wrote: > > > >> Hello All, > >> > >> I need to acquire packets from a source at more than 40Gbps. I was thi= nking > >> to use rx_burst in a function with more than one core and load it into= a > >> ring. My question is, will the packets received will be in order betwe= en > >> the cores say core 1 receives packet 1,2,3 and core 2 receives packet = 4,5,6 > >> or will it be in out of order(Random). Also can we combine 2 cores to > >> launch one function. As an alternative, you could have one lcore (dataplane thread) running the rte_eth_rx_burst() function in a loop (without locks), enqueueing all mbufs into a rte_ring ringbuffer. That allows capture of single-flow at very high= MPPS rates, depending on packet size, this likely covers 40Gbps. Later, multiple cores can consume from the ringbuffer (in Multi-Consumer mo= de) and each can write its own packet data to storage. If you need to be able to restore full ordering, the RX lcore could increme= nt a counter for each packet received. By storing that metadata with the packe= t absolute packet ordering could be restored for the entire packet stream, ev= en if multiple cores are actually writing each packet to separate storage file= s/devices. I'd likely take this design approach if your avg packet size is ~128 or hig= her. If its 64 byte packet line-rate you want, that's 59.x mpps, which is also a= chievable on one lcore, but perhaps using RSS with multiple RX lcores is a better approach (if entr= opy available). This becomes more difficult with absolute packet ordering, as multiple RSS = queues will ultimately be "out of absolute order" by nature of being in different = HW queues. (Restoring per-RSS-queue order is possible using the above counter approach= though :) > >> Thank you, > >> > >> With Regards, > >> Nirmalraj R Hope that helps! Regards, -Harry > > DPDK drivers assume a single core at a time is reading a single queue. > > If you have multiple cores reading a single queue then you need to do > > some form of mutual exclusion in your application. > > > > This is in the documentation