From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 163055A30 for ; Wed, 13 May 2015 11:03:23 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 13 May 2015 02:03:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,420,1427785200"; d="scan'208";a="492988077" Received: from irsmsx108.ger.corp.intel.com ([163.33.3.3]) by FMSMGA003.fm.intel.com with ESMTP; 13 May 2015 02:03:19 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.73]) by IRSMSX108.ger.corp.intel.com ([169.254.11.59]) with mapi id 14.03.0224.002; Wed, 13 May 2015 10:03:18 +0100 From: "Ananyev, Konstantin" To: Paul Emmerich Thread-Topic: [dpdk-dev] TX performance regression caused by the mbuf cachline split Thread-Index: AQHQjDpfvNvAYxOiB0uS83U6j3sEj513WF6AgAE+21A= Date: Wed, 13 May 2015 09:03:18 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772582142EB46@irsmsx105.ger.corp.intel.com> References: <554FF482.9080103@net.in.tum.de> <55512DE5.7010800@net.in.tum.de> <555138C7.5010002@net.in.tum.de> In-Reply-To: <555138C7.5010002@net.in.tum.de> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] TX performance regression caused by the mbuf cachline split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 09:03:24 -0000 DQpIaSBQYXVsLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGRldiBb bWFpbHRvOmRldi1ib3VuY2VzQGRwZGsub3JnXSBPbiBCZWhhbGYgT2YgUGF1bCBFbW1lcmljaA0K PiBTZW50OiBUdWVzZGF5LCBNYXkgMTIsIDIwMTUgMTI6MTkgQU0NCj4gVG86IGRldkBkcGRrLm9y Zw0KPiBTdWJqZWN0OiBSZTogW2RwZGstZGV2XSBUWCBwZXJmb3JtYW5jZSByZWdyZXNzaW9uIGNh dXNlZCBieSB0aGUgbWJ1ZiBjYWNobGluZSBzcGxpdA0KPiANCj4gRm91bmQgYSByZWFsbHkgc2lt cGxlIHNvbHV0aW9uIHRoYXQgYWxtb3N0IHJlc3RvcmVzIHRoZSBvcmlnaW5hbA0KPiBwZXJmb3Jt YW5jZToganVzdCBhZGQgYSBwcmVmZXRjaCBvbiBhbGxvYy4gRm9yIHNvbWUgcmVhc29uLCBJIGFz c3VtZWQNCj4gdGhhdCB0aGlzIHdhcyBhbHJlYWR5IGRvbmUgc2luY2UgdGhlIHRyb3VibGVzb21l IGNvbW1pdCBJIGludmVzdGlnYXRlZA0KPiBtZW50aW9uZWQgc29tZXRoaW5nIGFib3V0IHByZWZl dGNoaW5nLi4uIEkgZ3Vlc3MgdGhlIGNvbW1pdCByZWZlcnJlZCB0bw0KPiB0aGUgaGFyZHdhcmUg cHJlZmV0Y2hlciBpbiB0aGUgQ1BVLg0KPiANCj4gQWRkaW5nIGFuIGV4cGxpY2l0IHByZWZldGNo IGNvbW1hbmQgaW4gdGhlIG1idWYgYWxsb2MgZnVuY3Rpb24gZ2l2ZXMgYQ0KPiB0aHJvdWdocHV0 IG9mIDEyLjcvMTAuMzUgTXBwcyBpbiBteSBiZW5jaG1hcmsgd2l0aCB0aGUNCj4gc2ltcGxlL2Z1 bGwtZmVhdHVyZWQgdHggcGF0aC4NCj4gDQo+IERQREsgMS43LjEgd2FzIGF0IDE0LjEvMTAuNyBN cHBzLiBJIGd1ZXNzIEkgY2FuIGxpdmUgd2l0aCB0aGF0LCBzaW5jZQ0KPiBJJ20gcHJpbWFyaWx5 IGludGVyZXN0ZWQgaW4gdGhlIGZ1bGwtZmVhdHVyZWQgcGF0aCBhbmQgdGhlIGRyb3AgZnJvbQ0K PiAxMC43IHRvIH4xMC40IHdhcyBkdWUgdG8gYW5vdGhlciBjaGFuZ2UuDQo+IA0KPiBQYXRjaDog aHR0cHM6Ly9naXRodWIuY29tL2RwZGstb3JnL2RwZGsvcHVsbC8yDQo+IEkgYWxzbyBzZW50IGFu IGVtYWlsIHRvIHRoZSBtYWlsaW5nIGxpc3QuDQo+IA0KPiBJIGFsc28gdGhpbmsgdGhhdCB0aGUg cngtcGF0aCBjb3VsZCBhbHNvIGJlbmVmaXQgZnJvbSBwcmVmZXRjaGluZyBzb21ld2hlcmUuDQo+ IA0KDQpCZWZvcmUgc3RhcnQgdG8gZGlzY3VzcyB5b3VyIGZpbmRpbmdzLCB0aGVyZSBpcyBvbmUg dGhpbmcgaW4geW91ciB0ZXN0IGFwcCB0aGF0IGxvb2tzIHN0cmFuZ2UgdG8gbWU6DQpZb3UgdXNl IEJBVENIX1NJWkU9PTY0IGZvciBUWCBwYWNrZXRzLCBidXQgeW91ciBtZW1wb29sIGNhY2hlX3Np emU9PTMyLg0KVGhpcyBpcyBub3QgcmVhbGx5IGEgZ29vZCBjaG9pY2UsIGFzIGl0IG1lYW5zIHRo YXQgZm9yIGVhY2ggaXRlcmF0aW9uIHlvdXIgbWVtcG9vbCBjYWNoZSB3aWxsIGJlIGV4aGF1c3Rl ZCwNCmFuZCB5b3UnbGwgZW5kdXAgZG9pbmcgcmluZ19kZXF1ZXVlKCkuDQpJJ2Qgc3VnZ2VzdCB5 b3UgdXNlIHNvbWV0aGluZyBsaWtlICcgMiAqIEJBVENIX1NJWkUnIGZvciBtZW1wb29scyBjYWNo ZSBzaXplLA0KdGhhdCBzaG91bGQgaW1wcm92ZSB5b3VyIG51bWJlcnMgKGF0IGxlYXN0IGl0IGRp ZCB0byBtZSkuIA0KDQpBYm91dCB0aGUgcGF0Y2g6DQpTbyBmcm9tIHdoYXQgeW91IGFyZSBzYXlp bmcgLSB0aGUgcmVhc29uIGZvciB0aGUgZHJvcCBpcyBub3QgYWN0dWFsbHkgdGhlIFRYIHBhdGgs DQpidXQgcnRlX3BrdG1idWZfYWxsb2MoKS0+cnRlX3BrdG1idWZfcmVzZXQoKS4gDQpUaGF0IG1h a2VzIHNlbnNlIC0gIHBrdG1idWZfcmVzZXQoKSBub3cgaGFzIHRvIHVwZGF0ZSAyIGNhY2hlIGxp bmUgaW5zdGVhZCBvZiBvbmUuDQogRnJvbSBvdGhlciBzaWRlIC0gcnRlX3BrdG1idWZfYWxsb2Mo KSB3YXMgbmV2ZXIgY29uc2lkZXJlZCBhcyBhIGZhc3Rlc3QgcGF0aA0KKG91ciBSWC9UWCByb2l0 aW5pZXMgZG9uJ3QgdXNlIGl0KSAtIHNvIHdlIG5ldmVyIHB1dCBhIGJpZyBlZmZvcnQgaW4gdHJ5 aW5nIHRvIG9wdGltaXNlIGl0Lg0KDQpUaG91Z2gsIEkgYW0gcmVhbGx5IG5vdCBhIGJpZyBmYW4g b2YgbWFudWFsIHByZWZldGNoaW5nLiANCkl0cyBwYXJ0aWN1bGFyIGJlaGF2aW91ciBtYXkgdmFy eSAgZnJvbSBvbmUgY3B1IHRvIGFub3RoZXIsDQphbmQgaXMgcmVhbCBlZmZlY3QgaXMgc29ydCBv ZiBoYXJkIHRvIHByZWRpY3QsDQppbiBzb21lIGNhc2VzIGNhbiBldmVuIGNhdXNlIGEgcGVyZm9y bWFuY2UgZGVncmFkYXRpb24uDQpMZXQgc2F5IG9uIG15IElWQiBib3gsIHlvdXIgcGF0Y2ggZGlk bid0IHNob3cgYW55IGRpZmZlcmVuY2UgYXQgYWxsLg0KU28gSSB0aGluayB0aGF0ICdwcmVmZXRj aCcgc2hvdWxkIGJlIHVzZWQgb25seSB3aGVuIGl0IHJlYWxseSBnaXZlcyBncmVhdCBwZXJmb3Jt YW5jZSBib29zdA0KYW5kIHNhbWUgcmVzdWx0cyBjYW4ndCBiZSBhY2hpZXZlZCBieSBvdGhlciBt ZXRob2RzLiAgDQpGb3IgdGhhdCBwYXJ0aWN1bGFyIGNhc2UgLSBhdCBsZWFzdCB0aGF0ICdwcmVm ZXRjaCcgc2hvdWxkIGJlIG1vdmVkIGZyb20gX19ydGVfbWJ1Zl9yYXdfYWxsb2MoKQ0KdG8gIHJ0 ZV9wa3RtYnVmX2FsbG9jKCksIHRvIGF2b2lkIGFueSBuZWdhdGl2ZSBpbXBhY3Qgb24gUlggcGF0 aC4NClRob3VnaCwgSSBzdXBwb3NlIHRoYXQgc2NlbmFyaW8gbWlnaHQgYmUgaW1wcm92ZWQgd2l0 aG91dCBtYW51YWwgJ3ByZWZldGNoJyAtIGJ5IHJlb3JkZXJpbmcgY29kZSBhIGJpdC4NCkJlbG93 IGFyZSAyIHNtYWxsIHBhdGNoZXMsIHRoYXQgaW50cm9kdWNlIHJ0ZV9wa3RtYnVmX2J1bGtfYWxs b2MoKSBhbmQgbW9kaWZpZXMgeW91ciB0ZXN0IGFwcCB0byB1c2UgaXQuDQpDb3VsZCB5b3UgZ2l2 ZSBpdCBhIHRyeSBhbmQgc2VlIHdvdWxkIGl0IGhlbHAgdG8gY2xvc2UgYSBnYXAgYmV0d2VlbiAx LjcuMSBhbmQgMi4wPw0KSSBkb24ndCBoYXZlIGJveCB3aXRoIHRoZSBzYW1lIG9mZi1oYW5kLCBi dXQgb24gbXkgSVZCIGJveCByZXN1bHRzIGFyZSBxdWl0ZSBwcm9taXNpbmc6DQpvbiAxLjIgR0h6 IGZvciBzaW1wbGVfdHggdGhlcmUgaXMgcHJhY3RpY2FsbHkgbm8gZGlmZmVyZW5jZSBpbiByZXN1 bHRzICgtMC4zMyUpLCANCmZvciBmdWxsX3R4IHRoZSBkcm9wIHJlZHVjZWQgdG8gMiUuDQpUaGF0 J3MgY29tcGFyaW5nIERQREsxLjcuMSt0ZXN0cGFwcCB3aXRoIGNhY2hlX3NpemU9MipiYXRjaF9z aXplIHZzDQpsYXRlc3QgRFBESysgdGVzdHBhcHAgd2l0aCBjYWNoZV9zaXplPTIqYmF0Y2hfc2l6 ZStidWxrX2FsbG9jLg0KDQpUaGFua3MNCktvbnN0YW50aW4NCg0KcGF0Y2gxOg0KDQpkaWZmIC0t Z2l0IGEvbGliL2xpYnJ0ZV9tYnVmL3J0ZV9tYnVmLmggYi9saWIvbGlicnRlX21idWYvcnRlX21i dWYuaA0KaW5kZXggYWI2ZGU2Ny4uMjNkNzljYSAxMDA2NDQNCi0tLSBhL2xpYi9saWJydGVfbWJ1 Zi9ydGVfbWJ1Zi5oDQorKysgYi9saWIvbGlicnRlX21idWYvcnRlX21idWYuaA0KQEAgLTgxMCw2 ICs4MTAsNDUgQEAgc3RhdGljIGlubGluZSBzdHJ1Y3QgcnRlX21idWYgKnJ0ZV9wa3RtYnVmX2Fs bG9jKHN0cnVjdCBydGVfbWVtcG9vbCAqbXApDQogICAgICAgIHJldHVybiAobSk7DQogfQ0KDQor c3RhdGljIGlubGluZSBpbnQNCitydGVfcGt0bWJ1Zl9idWxrX2FsbG9jKHN0cnVjdCBydGVfbWVt cG9vbCAqbXAsIHN0cnVjdCBydGVfbWJ1ZiAqKm0sIHVpbnQzMl90IG4pDQorew0KKyAgICAgICBp bnQzMl90IHJjOw0KKyAgICAgICB1aW50MzJfdCBpOw0KKw0KKyAgICAgICByYyA9IHJ0ZV9tZW1w b29sX2dldF9idWxrKG1wLCAodm9pZCAqKiltLCBuKTsNCisNCisgICAgICAgaWYgKHJjID09IDAp IHsNCisgICAgICAgICAgICAgICBpID0gMDsNCisgICAgICAgICAgICAgICBzd2l0Y2ggKG4gJSA0 KSB7DQorICAgICAgICAgICAgICAgd2hpbGUgKGkgIT0gbikgew0KKyAgICAgICAgICAgICAgICAg ICAgICAgY2FzZSAwOg0KKyAgICAgICAgICAgICAgICAgICAgICAgUlRFX01CVUZfQVNTRVJUKHJ0 ZV9tYnVmX3JlZmNudF9yZWFkKG1baV0pID09IDApOw0KKyAgICAgICAgICAgICAgICAgICAgICAg cnRlX21idWZfcmVmY250X3NldChtW2ldLCAxKTsNCisgICAgICAgICAgICAgICAgICAgICAgIHJ0 ZV9wa3RtYnVmX3Jlc2V0KG1baV0pOw0KKyAgICAgICAgICAgICAgICAgICAgICAgaSsrOw0KKyAg ICAgICAgICAgICAgICAgICAgICAgY2FzZSAzOg0KKyAgICAgICAgICAgICAgICAgICAgICAgUlRF X01CVUZfQVNTRVJUKHJ0ZV9tYnVmX3JlZmNudF9yZWFkKG1baV0pID09IDApOw0KKyAgICAgICAg ICAgICAgICAgICAgICAgcnRlX21idWZfcmVmY250X3NldChtW2ldLCAxKTsNCisgICAgICAgICAg ICAgICAgICAgICAgIHJ0ZV9wa3RtYnVmX3Jlc2V0KG1baV0pOw0KKyAgICAgICAgICAgICAgICAg ICAgICAgaSsrOw0KKyAgICAgICAgICAgICAgICAgICAgICAgY2FzZSAyOg0KKyAgICAgICAgICAg ICAgICAgICAgICAgUlRFX01CVUZfQVNTRVJUKHJ0ZV9tYnVmX3JlZmNudF9yZWFkKG1baV0pID09 IDApOw0KKyAgICAgICAgICAgICAgICAgICAgICAgcnRlX21idWZfcmVmY250X3NldChtW2ldLCAx KTsNCisgICAgICAgICAgICAgICAgICAgICAgIHJ0ZV9wa3RtYnVmX3Jlc2V0KG1baV0pOw0KKyAg ICAgICAgICAgICAgICAgICAgICAgaSsrOw0KKyAgICAgICAgICAgICAgICAgICAgICAgY2FzZSAx Og0KKyAgICAgICAgICAgICAgICAgICAgICAgUlRFX01CVUZfQVNTRVJUKHJ0ZV9tYnVmX3JlZmNu dF9yZWFkKG1baV0pID09IDApOw0KKyAgICAgICAgICAgICAgICAgICAgICAgcnRlX21idWZfcmVm Y250X3NldChtW2ldLCAxKTsNCisgICAgICAgICAgICAgICAgICAgICAgIHJ0ZV9wa3RtYnVmX3Jl c2V0KG1baV0pOw0KKyAgICAgICAgICAgICAgICAgICAgICAgaSsrOw0KKyAgICAgICAgICAgICAg IH0NCisgICAgICAgICAgICAgICB9DQorICAgICAgIH0NCisNCisgICAgICAgcmV0dXJuIHJjOw0K K30NCisNCiAvKioNCiAgKiBBdHRhY2ggcGFja2V0IG1idWYgdG8gYW5vdGhlciBwYWNrZXQgbWJ1 Zi4NCiAgKg0KDQpwYXRjaDI6DQpkaWZmIC0tZ2l0IGEvbWFpbi5jIGIvbWFpbi5jDQppbmRleCAy YWE5ZmNmLi43NDljNTJjIDEwMDY0NA0KLS0tIGEvbWFpbi5jDQorKysgYi9tYWluLmMNCkBAIC03 MSw3ICs3MSw3IEBAIHN0YXRpYyBzdHJ1Y3QgcnRlX21lbXBvb2wqIG1ha2VfbWVtcG9vbCgpIHsN CiAgICAgICAgc3RhdGljIGludCBwb29sX2lkID0gMDsNCiAgICAgICAgY2hhciBwb29sX25hbWVb MzJdOw0KICAgICAgICBzcHJpbnRmKHBvb2xfbmFtZSwgInBvb2wlZCIsIF9fc3luY19mZXRjaF9h bmRfYWRkKCZwb29sX2lkLCAxKSk7DQotICAgICAgIHJldHVybiBydGVfbWVtcG9vbF9jcmVhdGUo cG9vbF9uYW1lLCBOQl9NQlVGLCBNQlVGX1NJWkUsIDMyLA0KKyAgICAgICByZXR1cm4gcnRlX21l bXBvb2xfY3JlYXRlKHBvb2xfbmFtZSwgTkJfTUJVRiwgTUJVRl9TSVpFLCAyICogQkFUQ0hfU0la RSwNCiAgICAgICAgICAgICAgICBzaXplb2Yoc3RydWN0IHJ0ZV9wa3RtYnVmX3Bvb2xfcHJpdmF0 ZSksDQogICAgICAgICAgICAgICAgcnRlX3BrdG1idWZfcG9vbF9pbml0LCBOVUxMLA0KICAgICAg ICAgICAgICAgIHJ0ZV9wa3RtYnVmX2luaXQsIE5VTEwsDQpAQCAtMTEzLDEzICsxMTMsMjEgQEAg c3RhdGljIHVpbnQzMl90IHNlbmRfcGt0cyh1aW50OF90IHBvcnQsIHN0cnVjdCBydGVfbWVtcG9v bCogcG9vbCkgew0KICAgICAgICAvLyBhbGxvYyBidWZzDQogICAgICAgIHN0cnVjdCBydGVfbWJ1 ZiogYnVmc1tCQVRDSF9TSVpFXTsNCiAgICAgICAgdWludDMyX3QgaTsNCisgICAgICAgaW50MzJf dCByYzsNCisNCisgICAgICAgcmMgPSBydGVfcGt0bWJ1Zl9idWxrX2FsbG9jKHBvb2wsIGJ1ZnMs IFJURV9ESU0oYnVmcykpOw0KKyAgICAgICBpZiAocmMgPCAwKSB7DQorICAgICAgICAgICAgICAg UlRFX0xPRyhFUlIsIFVTRVIxLA0KKyAgICAgICAgICAgICAgICAgICAgICAgIiVzOiBydGVfcGt0 bWJ1Zl9hbGxvYyglenUpIHJldHVybnMgZXJyb3IgY29kZTogJWRcbiIsDQorICAgICAgICAgICAg ICAgICAgICAgICBfX2Z1bmNfXywgUlRFX0RJTShidWZzKSwgcmMpOw0KKyAgICAgICAgICAgICAg IHJldHVybiAwOw0KKyAgICAgICB9DQorDQogICAgICAgIGZvciAoaSA9IDA7IGkgPCBCQVRDSF9T SVpFOyBpKyspIHsNCi0gICAgICAgICAgICAgICBzdHJ1Y3QgcnRlX21idWYqIGJ1ZiA9IHJ0ZV9w a3RtYnVmX2FsbG9jKHBvb2wpOw0KLSAgICAgICAgICAgICAgIHJ0ZV9wa3RtYnVmX2RhdGFfbGVu KGJ1ZikgPSA2MDsNCi0gICAgICAgICAgICAgICBydGVfcGt0bWJ1Zl9wa3RfbGVuKGJ1ZikgPSA2 MDsNCi0gICAgICAgICAgICAgICBidWZzW2ldID0gYnVmOw0KKyAgICAgICAgICAgICAgIHJ0ZV9w a3RtYnVmX2RhdGFfbGVuKGJ1ZnNbaV0pID0gNjA7DQorICAgICAgICAgICAgICAgcnRlX3BrdG1i dWZfcGt0X2xlbihidWZzW2ldKSA9IDYwOw0KICAgICAgICAgICAgICAgIC8vIHdyaXRlIHNlcSBu dW1iZXINCi0gICAgICAgICAgICAgICB1aW50NjRfdCogcGt0ID0gcnRlX3BrdG1idWZfbXRvZChi dWYsIHVpbnQ2NF90Kik7DQorICAgICAgICAgICAgICAgdWludDY0X3QqIHBrdCA9IHJ0ZV9wa3Rt YnVmX210b2QoYnVmc1tpXSwgdWludDY0X3QqKTsNCiAgICAgICAgICAgICAgICBwa3RbMF0gPSBz ZXErKzsNCiAgICAgICAgfQ0KICAgICAgICAvLyBzZW5kIHBrdHMNCg0K