From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0045.outbound.protection.outlook.com [104.47.41.45]) by dpdk.org (Postfix) with ESMTP id 723167CF9 for ; Fri, 27 Apr 2018 03:29:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=studentuml.onmicrosoft.com; s=selector1-student-uml-edu; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=QpzyamyNIp7Djpv/HhvEpE+7COI1Qdqv5enIuS824TE=; b=kyPewDlKWXsY6Mao7inqMSVwsKdZlp73mlbFRh9JxUyhEW5jiCucaa1pi+WidOCriVZlYfP+ozx++UUudXuv+6Qw19BzxU5tWmb3t/m2DQ7AG6u36dW/rGl8i6Bhev1aN05IqQNw42hrQfSzYIaI0XsMQxszsAP0GU2Jkvq+dRs= Received: from DM5PR02MB2523.namprd02.prod.outlook.com (10.168.195.17) by DM5PR02MB3832.namprd02.prod.outlook.com (52.132.139.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Fri, 27 Apr 2018 01:29:38 +0000 Received: from DM5PR02MB2523.namprd02.prod.outlook.com ([fe80::f8d0:e60a:5495:6fd2]) by DM5PR02MB2523.namprd02.prod.outlook.com ([fe80::f8d0:e60a:5495:6fd2%13]) with mapi id 15.20.0696.020; Fri, 27 Apr 2018 01:29:38 +0000 From: "Wu, Xiaoban" To: Stephen Hemminger CC: "users@dpdk.org" Thread-Topic: [dpdk-users] Interesting considerate but annoying behavior of MLX5 driver Thread-Index: AQHT3arn+5BHs4kFF0y+3oUgfrSmWqQTo/CAgAAuwOc= Date: Fri, 27 Apr 2018 01:29:38 +0000 Message-ID: References: , <20180426154055.16842d96@xeon-e3> In-Reply-To: <20180426154055.16842d96@xeon-e3> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Xiaoban_Wu@student.uml.edu; x-originating-ip: [129.63.5.153] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DM5PR02MB3832; 7:x6KvyakFlbQ8FP2l/MCO6v+QsGJeQQBJC0XspWgWxpErva96zsfam51B7xJuv0cIywnkkNDmGpjGYYjOxFsXSI3mWSnEgfn4r4SVeJawuqWcDsHzNTCSquAM9TpAntMe8BzaRVqWXbnOTBm6WdMpn59CDmOwyi3iS7JeddOossUgZTagVPQIYy4IpgtUMwg/J9RZUVAXxdkGc4PFfosu2F67RaBG1AjkslnjJoHrD4PLjgxCsk4HN2u+uzr6E9jl x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:(274723738142955); BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:DM5PR02MB3832; x-ms-traffictypediagnostic: DM5PR02MB3832: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(274723738142955)(192823040165218)(269151656437849); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(3231232)(944501410)(52105095)(3002001)(93006095)(93001095)(6041310)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(6072148)(201708071742011); SRVR:DM5PR02MB3832; BCL:0; PCL:0; RULEID:; SRVR:DM5PR02MB3832; x-forefront-prvs: 0655F9F006 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39380400002)(396003)(39860400002)(376002)(366004)(346002)(189003)(199004)(7696005)(5660300001)(86362001)(2906002)(74316002)(3660700001)(59450400001)(53546011)(26005)(88552002)(53936002)(99286004)(6916009)(6506007)(229853002)(186003)(6246003)(316002)(97736004)(102836004)(486006)(786003)(575784001)(75432002)(11346002)(6606003)(476003)(7736002)(105586002)(66066001)(5250100002)(2900100001)(72206003)(3846002)(3280700002)(446003)(33656002)(6116002)(14454004)(68736007)(55016002)(81166006)(9686003)(25786009)(106356001)(19627405001)(8936002)(6436002)(8676002)(76176011)(478600001)(81156014)(54896002)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR02MB3832; H:DM5PR02MB2523.namprd02.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: student.uml.edu does not designate permitted sender hosts) x-microsoft-antispam-message-info: m58e/ZV+zRqC9A3ytPUXUXebOpkEdqZZgtP0tyl3YCvsD2u4ZwrunPmAe79ACvVigMvDpOnN06ckRib/jyBlo0QnBA9uOkbyYO6JbtX7EyzKpCZcD5Zc0nb4YSU4Xh2yJN4zyA0jtnggn4p+l3is8eWlAn6EBDaQF5KXGVr+bfY+qYfRo351lXxd1mquzWJT spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: d05b313e-d043-45eb-6f0c-08d5abde57f4 X-OriginatorOrg: student.uml.edu X-MS-Exchange-CrossTenant-Network-Message-Id: d05b313e-d043-45eb-6f0c-08d5abde57f4 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Apr 2018 01:29:38.8582 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4c25b8a6-17f7-46f9-83f0-54734ab81fb1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR02MB3832 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Interesting considerate but annoying behavior of MLX5 driver X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 01:29:41 -0000 Hi Stephen, Thanks very much for your help. Though I don't know if it is a overkill, th= e following does the job. /sbin/ethtool -s enp59s0f0 autoneg off /sbin/ethtool -A enp59s0f0 rx off /sbin/ethtool -A enp59s0f0 tx off Best wishes, Xiaoban ________________________________ From: Stephen Hemminger Sent: Thursday, April 26, 2018 6:40:55 PM To: Wu, Xiaoban Cc: users@dpdk.org Subject: Re: [dpdk-users] Interesting considerate but annoying behavior of = MLX5 driver On Thu, 26 Apr 2018 22:26:36 +0000 "Wu, Xiaoban" wrote: > Dear All, > > > I have two Dell PowerEdge R740 servers A and B running with ubuntu 16.04,= each one has a Mellanox MCX556A-ECAT NIC installed on the PCIe x16 slot. A= nd, the two NICs are directly connected back to back with a copper cable. > > > On server A, it runs a RX program which runs a function to process and an= alyze the received packets. On server B, it runs a packet-gen program which= generates and sends packets out. They are both compiled with dpdk-17.11 > > > Now the interesting thing is that on server B, it is kind of smart and co= nsiderate enough to automatically adjust its TX throughput according to how= fast server A processes the packets. If server A processes the packets fas= ter, then server B sends packets with a higher throughput. Similarly, if se= rver A processes the packets slower, then server B sends packets with a low= er throughput. Please note that once the program on server B starts, it is = never interrupted by any way. > > > However, I think the server B should send out the packets with a constant= throughput no matter how fast server A processes the packets. > > > Does anybody else notice this interesting behavior of MLX5 driver? Can an= ybody help me disable this feature permanently? Thanks very much for your h= elp. > > > Best wishes, > > Xiaoban Turn off Ethernet flow control. It was a design mistake that NIC vendors a= lways seem to like because it gives 0 loss benchmarks.