From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from DNVWSMAILOUT1.mcafee.com (dnvwsmailout1.mcafee.com [161.69.31.173]) by dpdk.org (Postfix) with ESMTP id 6CB492E41 for ; Mon, 17 Jul 2017 13:00:45 +0200 (CEST) Received: from MIVEXAPP1N01.corpzone.internalzone.com (unknown [10.48.48.88]) by DNVWSMAILOUT1.mcafee.com with smtp id 61fd_850a_b6bab6f9_bbd7_429e_863c_1085a53f22cc; Mon, 17 Jul 2017 06:00:43 -0500 Received: from MIVEXAPP1N02.corpzone.internalzone.com (10.48.48.89) by MIVEXAPP1N01.corpzone.internalzone.com (10.48.48.88) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Mon, 17 Jul 2017 07:00:42 -0400 Received: from MIVO365EDGE3.corpzone.internalzone.com (10.48.176.86) by MIVEXAPP1N02.corpzone.internalzone.com (10.48.48.89) with Microsoft SMTP Server (TLS) id 15.0.1263.5 via Frontend Transport; Mon, 17 Jul 2017 07:00:42 -0400 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (10.48.176.241) by edge.mcafee.com (10.48.176.86) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Mon, 17 Jul 2017 07:00:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mcafee.onmicrosoft.com; s=selector1-mcafee-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rJ+VoWxhlieu65p6yuGzVX5bWtPD9K3mzUr8sHRf51A=; b=QBcEnJIz/cEQPA2o0ulGBVg3i/JwnNcOGdw2flt0zi0BcnvOkxDkAomqT8DNszK8hXUn0JaRUjqTZGDk/4u12pT/FHYujht3hkkZinBdCDw7nVhwkLIvJuOKzLmrDv8D/u7tmvMUiruzEt8A19KGnM3HHB7CQQrBa9lwbCObCIE= Received: from DM3PR16MB0733.namprd16.prod.outlook.com (10.164.201.151) by MWHPR16MB1728.namprd16.prod.outlook.com (10.172.60.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1261.13; Mon, 17 Jul 2017 11:00:40 +0000 Received: from DM3PR16MB0733.namprd16.prod.outlook.com ([10.164.201.151]) by DM3PR16MB0733.namprd16.prod.outlook.com ([10.164.201.151]) with mapi id 15.01.1261.022; Mon, 17 Jul 2017 11:00:39 +0000 From: "Williams, Christopher" To: "users@dpdk.org" CC: "Neal, Peter" Thread-Topic: rte_eth_dev_rss_reta_update appears to have no effect with net_ixgbe driver Thread-Index: AdL+5w5gYvE0wjNzRLGVr5ko8IoE+Q== Date: Mon, 17 Jul 2017 11:00:39 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=McAfee.com; x-originating-ip: [161.69.163.20] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; MWHPR16MB1728; 7:i34Y8bFIVjyiYxHqbQVIclLan2yXbzDRDMC1PxqydvvSXHCV+2Ou230w4urqIiqvOwJEwKt3J7LvgKGlbxsEEbR6uAtIja9kdN1mwMyaKWfrAuhj1VpQhEEpk7cTmRdsXfnhjyCCkZsQ2rrWEmVZJ+IBDuMjCnfOvbNZaw3TJ9lPNnd8ltj64n6xhFSKCgoQ3Zjb+s0qGvYVWbdInNMRGOf/LYKo5h1mDg9bKV+u2vmCtEDdbT8ffEptqD6It72x12PmgOW/l1fn0JlwUfPUJcpfr2yA3KfSpIviyaebKVXdA+AFDnf+z9M6dhPkI/nlw1/oq9AtmRPhXbveJpYUHbia77i46p++V26w25HrT2hkJjuPj70SXvwa3MBnaI/AQerzWkYsI84OPQEyeSL0T4JR18r4gxyuViB/4Oj4qqQYUJ9Jevs3wp3ofjGqNvyhxY7CGC09Tz+jx0lGhWt42ROksjzdRMenaeTVps/0yDNPtCr/F2VtJKO3kAaQavWMZNd1KlTrdiLa8aiommcYIKgr5toZOfn8Vpgn8AF8k7r3dpeabjZK3XDJr+GhuyW8di38XsYgCrcG3968bTnigB6YpEcq27wqc8xBq2E3VnBrOy9J8jcQyCg8GYqk/bV3aGPv86fCEMrxNKgDlgRCKw/VY7bpElUzVTqTyVou//JrIzOWArhw5WbzeM1/nMHhq+9YCYbo+ruOChecW5ifIwOGnTAdB1jq0Fq3rFpl81+rp4/gaSg1kNzj2RNdL5MAQFxhfvkxYx/BWdiFMy/tI9r0L6+7NdZ3YUwQgrCXLQk= x-forefront-antispam-report: SFV:SKI; SCL:-1SFV:NSPM; SFS:(10009020)(6009001)(39850400002)(39400400002)(39840400002)(39410400002)(39450400003)(14454004)(66066001)(6506006)(2501003)(7696004)(33656002)(6436002)(54356999)(86362001)(50986999)(2351001)(5660300001)(25786009)(3280700002)(3660700001)(478600001)(72206003)(4326008)(38730400002)(107886003)(110136004)(81166006)(8936002)(7736002)(8676002)(1730700003)(99286003)(55016002)(80792005)(5640700003)(6116002)(3846002)(102836003)(74316002)(189998001)(2900100001)(77096006)(305945005)(9686003)(6916009)(2906002)(53936002)(85282002); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR16MB1728; H:DM3PR16MB0733.namprd16.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; x-ms-office365-filtering-correlation-id: 07d70580-6b9c-4bcc-afec-08d4cd030f8f x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:MWHPR16MB1728; x-ms-traffictypediagnostic: MWHPR16MB1728: x-exchange-antispam-report-test: UriScan:(158342451672863)(236129657087228)(247924648384137); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(2017060910075)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(3002001)(6041248)(20161123560025)(20161123564025)(20161123558100)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR16MB1728; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR16MB1728; x-forefront-prvs: 0371762FE7 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jul 2017 11:00:39.4034 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4943e38c-6dd4-428c-886d-24932bc2d5de X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR16MB1728 X-OriginatorOrg: mcafee.com X-NAI-Spam-Flag: NO X-NAI-Spam-Level: X-NAI-Spam-Threshold: 15 X-NAI-Spam-Score: 0.1 X-NAI-Spam-Version: 2.3.0.9418 : core <6072> : inlines <5978> : streams <1754565> : uri <2464533> Subject: [dpdk-users] rte_eth_dev_rss_reta_update appears to have no effect with net_ixgbe driver X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jul 2017 11:00:46 -0000 We're using receive side scaling with Intel NICs that use the net_ixgbe PMD= . Our application needs to be able to follow entire conversations in a higher= -level protocol, such as HTTP or SMTP, which requires us to set a hashing k= ey such that (a, b) hashes to the same value as (b, a) (i.e. client->server= packets and server->client packets must end up in the same queue). The req= uirement to be able to follow FTP conversations limits us to hashing over I= P addresses only, since any FTP conversation will use multiple TCP ports. Hashing IP addresses only results in extremely uneven distribution of packe= ts into the rx queues. We *should* be able to fix this by setting a redirec= tion table to bias packets towards one table rather than another. However, = configuring a redirection table appears to have no effect. The following C++ function is a slight simplification of what we're actuall= y doing: #define MAX_RETA_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) void config_eth_dev(uint8_t port_id, uint16_t nRx) { rte_eth_conf eth_config; memset(ð_config, 0, sizeof(eth_config)); struct rte_eth_dev_info inf; rte_eth_dev_info_get(port_id, &inf); eth_config.rxmode.mq_mode =3D ETH_MQ_RX_RSS; // Enable receive-side sca= ling // Set a hashing key such that frames from a->b and b->a are inserted i= nto // the same hardware queue static uint8_t hashKey[] =3D { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, }; eth_config.rx_adv_conf.rss_conf.rss_key =3D hashKey; eth_config.rx_adv_conf.rss_conf.rss_key_len =3D sizeof(hashKey); // Hash IP addresses only so that, e.g., FTP packets go to the same rx = queue eth_config.rx_adv_conf.rss_conf.rss_hf =3D ETH_RSS_IPV4 | ETH_RSS_IPV6; if (rte_eth_dev_configure(port_id, nRx, 1, ð_config)) { throw std::runtime_error("rte_eth_dev_configure"); } rte_eth_dev_info_get(port_id, &inf); // Set a redirection table - this should distribute traffic more-or-les= s evenly // to each queue uint16_t rSize =3D inf.reta_size; // RX queue distribution for two queues static uint16_t dist2[] =3D { 0, 0, 0, 0, 1, 1, 1, }; // RX queue distribution for four queues static uint16_t dist4[] =3D { 0, 0, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, }; uint16_t *pd =3D nullptr, i; size_t nd =3D 0; switch (nRx) { case 4: pd =3D dist4; nd =3D sizeof(dist4) / sizeof(uint16_t); break; case 2: pd =3D dist2; nd =3D sizeof(dist2) / sizeof(uint16_t); break; default: break; } if (rSize && rSize <=3D ETH_RSS_RETA_SIZE_512 && pd) { struct rte_eth_rss_reta_entry64 rConf[MAX_RETA_SIZE]; memset(rConf, 0, sizeof(rConf)); for (i =3D 0; i < rSize; i++) { rConf[i / RTE_RETA_GROUP_SIZE].mask =3D UINT64_MAX; } for (i =3D 0; i < rSize; i++) { uint16_t reta_id =3D i / RTE_RETA_GROUP_SIZE; uint16_t reta_pos =3D i % RTE_RETA_GROUP_SIZE; uint16_t rss_qs_pos =3D i % nd; rConf[reta_id].reta[reta_pos] =3D pd[rss_qs_pos]; } if (rte_eth_dev_rss_reta_update(port_id, rConf, rSize)) { throw std::runtime_error("rte_eth_dev_rss_reta_update"); } } } The code for configuring the redirection table is lifted from the ip_pipeli= ne sample application. Is there something we're doing wrong here (or something we're failing to do= )? Christopher Williams. ________________________________ McAfee Security UK Limited is registered in England and Wales with its regi= stered address at C/O Skadden, Arps, Slate, Meagher & Flom (UK) LLP, 40 Ban= k Street, Canary Wharf, London, United Kingdom, E14 5DS, Company No. 104728= 68