From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 785CAA0093; Thu, 10 Mar 2022 18:04:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 083DF4113F; Thu, 10 Mar 2022 18:04:36 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2082.outbound.protection.outlook.com [40.107.212.82]) by mails.dpdk.org (Postfix) with ESMTP id 88C634113E; Thu, 10 Mar 2022 18:04:34 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XgMXpG2dWa9A1TAxbiVPBUBvO0meyCnCEoFv96/BGlZRuxyJ1J3WFD+LgkcQi5cTqYdOf94u084JLV1Y0NRO0A44Qes5+KAtLwpTRRvQecAQmGCG3Cslf7Qf21ZarQNMBXeqq1HNmHZNlqEiD8BRisP4bSZV7h4QKmEMIxZvNM3+AVEk4e4sF9RU+sBXlo86vfEwnlQHRP+9uL6Sr+IL45aZ95DEwIiiX5Oh6U8eVDxmSlBeza0E+Dwn6qVzbihsJyNNaUSIxGgV/2LQqG3mz5E8C0VNbn3sIsSTC4YHkF2i7Y6WAfz3T55XetNLWeiZsGiyhk0zjP5CVllXyhMuYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nuT5LDTDJQTWDoM0nXYsG1nzdjwV9WFpNSzRBoaDjUk=; b=ZqgLyDdE1stWxh8dQQ01o+T5ohaeYNCDsYHD1pTCjPbSrPi+FLN5mDxW2CPAaRadXZBe9xC7IxTmd9MEPyfmnSrP7GMW14u2a+uGlb+SXNxuHoYc17YGJqgdrV8drBal9KpJjSpksh4x5KCB+7VPX67GTEVBPFnnh/rpy8dt0yxonRIfc5yFbOE98lmfvbXmN3k8X5eAM8hLZdNHbH3RwDS+qrtmQXNuXjJCssvZyBNr3TblnQS8EIOvm4twC0sajoQA9MpDR3/KJYecSsU/bdFrWlz8DtgZPqsq13d5mf1vro694gLO6vnSxjZgnhM5nZQWJEkeJb09uxUQVl/pTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nuT5LDTDJQTWDoM0nXYsG1nzdjwV9WFpNSzRBoaDjUk=; b=SGqsKz9E7M8IJ2I8TDAVelT6i/GMgHgy6aVlWqkaBpBgtbSQC+sfvBAaFsP/4KOMD4byxm+fv3KN7Em3nGAalwf+62etL8KHXLIEhXotJkR2GH/ADXeTr9Rn3JQWUxf18TUsDFZsZZOPMB2NJEX66lwrcyOp+2v627Xypau5lIRmtJ6agpCptDJwlBrqjsED4rA0Iq2qsmXryb+5Tx236T97Je0rRd3unVvtsoKF28vRd5SaHsNpZ1R2/jaHFO5i0RpRuVgoJDdP9SZsXnuCwidDfSk6id3fK4yXAMxwihZcTeB8sJGU2xaXYOEf3UXhHfdszQOyqtI0qL7AhNbT7Q== Received: from DM5PR12MB4661.namprd12.prod.outlook.com (2603:10b6:4:aa::32) by BN8PR12MB3139.namprd12.prod.outlook.com (2603:10b6:408:41::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.15; Thu, 10 Mar 2022 17:04:32 +0000 Received: from DM5PR12MB4661.namprd12.prod.outlook.com ([fe80::d1c7:72a3:d491:b6f]) by DM5PR12MB4661.namprd12.prod.outlook.com ([fe80::d1c7:72a3:d491:b6f%4]) with mapi id 15.20.5038.027; Thu, 10 Mar 2022 17:04:32 +0000 From: Michael Baum To: "NBU-Contact-Thomas Monjalon (EXTERNAL)" , David Marchand CC: "dev@dpdk.org" , Matan Azrad , Raslan Darawsheh , Slava Ovsiienko , dpdk stable Subject: RE: [PATCH] net/mlx5: fix FD configuration for Rx interrupt Thread-Topic: [PATCH] net/mlx5: fix FD configuration for Rx interrupt Thread-Index: AQHYNIGAbxsw5tpWFkWnD0CHihaAy6y4uV+AgAAR6ACAAAvPwA== Date: Thu, 10 Mar 2022 17:04:32 +0000 Message-ID: References: <20220310131923.1144368-1-michaelba@nvidia.com> <4694126.GXAFRqVoOG@thomas> In-Reply-To: <4694126.GXAFRqVoOG@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bfe2f011-2b9d-4928-9959-08da02b80be7 x-ms-traffictypediagnostic: BN8PR12MB3139:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: HNmF2aTmQZkXyH7X1jG7e8RJ9XG9IFqHa5+R+JkrLiP2n9BvB24QL81ZlWU3yt/CJahxpvqMpoLdRCvKR1/jrNwHt8XuIfmIY6DW5V54UqqXeIPkCyUHtXdKMVgOYwZFpEsgVU2P4Vw5pnKp0apObAZ6ThG+asPZskx7ja04spx1vcuq3QUtbhdydUGLjWSO2MKo72ImU5lKB26xHH8z1IvvzrR7AAuDzgZg4gnBR2TetnUdhBEWS0zKq4T5JWoITpc4TVhUNanDhLxwbtAp961EwuI/FKMhF9/gBJ6IEuhUrPnoUCwHZYDp8H2AYa6fVHRNQE1Qs7VMEH9xR4qRXvpgzcG2sFpZnf0QovtHjCfBFlfkV1srHLos8367xIbcezDtG5ohTezCzbQlevaqoOkzc1vVFPY2BJOUg1OZ7idvmuJ4Cq5hqT38tfA6oQ1uIjnMyR5TgT5fHX2WUce7V/jTp+46xvWXxVgFF7zHNg2+rk6Bbq9nJedHY4pgdDM15JIsxGcw4eU9NlP48z6CtPL+SEy8ry6gIaL8CUL2KSrwt2KkJopqMAAi1WTsOdj9Ixd7W8sfogkbvibjg019Y1AvmAJDhwv2kOCljYH4r+W7HFFRnQkojzh+gO/hHajrsy0CVez/KsQSEsjHsNmOQVGnfaAnRVYDuB4ZLtuiYzSPvMQBfAtnavaTK2hlyaPFfJjEUNciDEUgd3f1q1MWD40HiqOGVVNXg3+7L21+TM61QPMgmkAWXYsxXAvpoWj4hWnXrJqbSuG1+Ql4MXq+eyHGqLzFRhVEHOzH993feRg= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR12MB4661.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(86362001)(52536014)(8936002)(316002)(76116006)(38070700005)(66556008)(66446008)(66476007)(64756008)(966005)(110136005)(66946007)(54906003)(45080400002)(55016003)(2906002)(9686003)(7696005)(53546011)(6506007)(5660300002)(33656002)(71200400001)(26005)(186003)(508600001)(83380400001)(122000001)(38100700002)(8676002)(4326008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CO+vRpYg8gxBRObHVYQata9hn4JkiE7X8AU+6g4KO+QJRpdDmu+Wxp/QpP3+?= =?us-ascii?Q?a+CUA+V4WQh/JOc/mJrWmZUF+b0VudHivi16JpmlwAstbb8Y+XD6BUinVqCQ?= =?us-ascii?Q?kZHLmhJJ9avVn7B4xN29+YA5h2z1s/KUvKiLtTFx1xffGN7zVn+vUIIROc4R?= =?us-ascii?Q?oApnQ/hEfNy2LYGiwnQv11IOVdm1MkCjF2ZXEmwFBqpHVyRzzElZwHHAvQ7y?= =?us-ascii?Q?WuLRirD26cfnvvMJygIPmvBQfarNp/WoyzIriIXFNXa5RJj0NgxcxPgmcICf?= =?us-ascii?Q?MG5ozLXrNwBNuQsJ9UgfbA1cWjzRTICVxFM/zXd4qMHYDySfFIMj1jmoC/wd?= =?us-ascii?Q?iWMUysEb7GQMsby9GdOE5tKvg/Bmgi07DtLEKC4R/tk9Tbor1sDKFc9d983e?= =?us-ascii?Q?Vu/CmOlys/UjNi4jedD15hh3ZpzlCJZwQYwctnLaymioPSLJAHAMomn2bhXM?= =?us-ascii?Q?4gIVGLAXJ8ddCaNSFELOmIb1S3ujeTPRVtP1m18FE+9s3CzH68J7XHhu01dH?= =?us-ascii?Q?4g17CL3HnDuTZ12HWO6zQscSLeBLZKLpEe6i9ecwyDqqj+uuFqo6+s9pgeWk?= =?us-ascii?Q?FWmL656FdnWk+pU3LKxrXQrGT6GtQ7edboOHn8cfbVDPNRVRJZ22UOgUmr5r?= =?us-ascii?Q?qfKDbRmtsYMZhnOhy65kJ76ZcCPjfIXnmeOG/vfpzS/TJyPwJr+4LRUNG6GB?= =?us-ascii?Q?3alDh9OOC9+bbRhMy0z/cjdu2G2SI8GcqzOBD9FS0VvHBxz9lC4pq4GN7sx4?= =?us-ascii?Q?a5+aw6ghtJofpjT8aPw+RnZknwfrtAtDPLRNSigHX2nTEwr5i+JDFBHytZCS?= =?us-ascii?Q?XcKuuAsKQzx6HZPBDSdgR0XLHFlBvQabmExnbman2G98OelNwARgj4uJZsZp?= =?us-ascii?Q?TdmY7SI/24uSgjqwTtTAq4Gunb92X+DIdq5AdT5F0zl7nwNxN1kOtaK0Q7Ui?= =?us-ascii?Q?SjMJEURnU0G4mTpabU+hcaaPv5InNRYXox3tIcAvjaW68zwJRrVU3z4kiJek?= =?us-ascii?Q?V6Zbeajzoj7TjqIdg7fM/QKcIxcXbpNd4ZdaeHIhHIVM6rYSj6UbJVdbM6GG?= =?us-ascii?Q?+H8u4QJ/wcEqHl9NgpUGRUAdObScNz7zWqqwSnGZvoL5q4PhxM7rQ894IGHQ?= =?us-ascii?Q?bWr5lEiM8X5G8742nOaQ8n7MVQurvbazxPD1K80r8Dspa0p0qkhWJsH6capp?= =?us-ascii?Q?94wwIfebMsN81MVZPNejzRNDbRQEYAAPph3l36+wcPPXUFsAxWiwtwT8u62O?= =?us-ascii?Q?I0Zk8N/f8dw95ALm2szIe3lG7gVBmbZ9iM2UQWFJ+1/2t9t9DlNSetTvmnxK?= =?us-ascii?Q?VjwxGlFupSIAsmAQqabz8pcdJdlriZCy6JodDGkCVYAw+OeGng+Zzy9VK/Pt?= =?us-ascii?Q?yU5pAhlBY/2X2JJr2TFzdS8KVnaWOxTxdRkoIk0OSogGDAZtWMvyLioK+tgZ?= =?us-ascii?Q?wp1rKGbWEUiehVaD8DMmUCUsVRZTW22/MEjVropfNladr0uxybwF8SDiB3Y/?= =?us-ascii?Q?KLhJCA5gxOYwBPguJCfpBR5ECB3iiJwRYdRNPycKcqdx3FmsjMy6LtJp1JhI?= =?us-ascii?Q?RkAVxZIox17aope2BqtldlDzfGmD0m84BHZVM6ChksompdQ0yaME4czMf1EY?= =?us-ascii?Q?J0hZKhkjfmOrI/fJYKMAOeY=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB4661.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bfe2f011-2b9d-4928-9959-08da02b80be7 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Mar 2022 17:04:32.2062 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 7KKdyxDuHGhsppw7hPZRKGqSZO/fo+sLnlqckrxqpFnzz3tGfEwcsdunUFvPltX1bGAAG91SBlBXwZ9tFrkrlA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB3139 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Mar 10, 2022 at 6:16 PM Thomas Monjalon wrote= :=20 >=20 > 10/03/2022 16:12, David Marchand: > > On Thu, Mar 10, 2022 at 2:19 PM Michael Baum > wrote: > > > > > > The mlx5_rx_intr_vec_enable() function allocates queue vector and > > > fill FD list for Rx interrupts. > > > > > > The driver wrongly configured the FD with a non-blocking flag which > > > prevent waiting on this FD. > > > > > > This patch removes O_NONBLOCK flag adding. > > > > - Maybe I deserve a Reported-by: credit on this issue. Hi David, You are right, you should have get the credit. > > I sent a proposal to make use of Rx interrupts in OVS > > > https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fpatc > hwork.ozlabs.org%2Fproject%2Fopenvswitch%2Fpatch%2F20220304161132.2 > 2065-1- > david.marchand%40redhat.com%2F&data=3D04%7C01%7Cmichaelba%40n > vidia.com%7C3679e1e70986425d465b08da02b14b34%7C43083d15727340c1b7 > db39efd9ccc17a%7C0%7C0%7C637825257758477604%7CUnknown%7CTWFpb > GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI > 6Mn0%3D%7C3000&sdata=3DLamFMZTLgZzQoUDBYAGn2aL9z7RWylWCkn > q2SGFuKOY%3D&reserved=3D0. > > And that's when I noticed that mlx5 rx fds were waking OVS too often > > and reported it to mlx5 maintainers. > > > > > > - Testing this patch by starting dpdk-l3fwd-power example (and no > > traffic sent at all): > > > > # strace -r -f ./dpdk-dir/v21.11/examples/dpdk-l3fwd-power --lcores > > 0@3,1@5 -a 0000:82:00.0 --in-memory -- -p 0x1 -P --config '(0,0,1)' > > ... > > [pid 534983] 0.000348 epoll_wait(26, [], 1, 10) =3D 0 > > [pid 534983] 0.010082 read(24, > > > > For some reason, there is an event available for fd 18 right away > > (which is the issue in the first place). I don't understand who is FD 18, I cannot see it in your log > > When reading this fd, read() blocks until an actual packet is received. > > > > Then, I send exactly one packet: > > [pid 534983] 0.010082 read(24, "@\217:\370\21\0\0\0", 136) =3D 8 > > [pid 534983] 9.228478 epoll_wait(26, [], 1, 10) =3D 0 > > [pid 534983] 0.010082 read(24, > > > > That makes mlx5 rx interrupts unusable for an application that does > > more than just polling one rxq. >=20 > Excuse me, I don't understand this trace. > What is the first read? > Having a second read after epoll_wait is normal if a packet is received, = isn't it? >=20