From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20046.outbound.protection.outlook.com [40.107.2.46]) by dpdk.org (Postfix) with ESMTP id 8B5BB2B8C for ; Mon, 12 Nov 2018 06:25:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dQn8KMAwoyQ5LrSogRTtt+q4Os9TROapoZ0Yp3qcz6Y=; b=a6cbkh5o7jiZJIHmCpV0Cvmu2TlRZD0i0DK1KeMtobFhRrdBeIOBA0380NK3zV67dsZ1mnX190WSzb9CV1XLHDeSVLdX6vKEESxfCyya6ev3J340g7qMKIfKA/YnrlLNRZAGVeGGPidhtl6Lmn5NEArg1LzeiSQHCtyirh7suO0= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.186.150) by AM4PR05MB3460.eurprd05.prod.outlook.com (10.171.187.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.26; Mon, 12 Nov 2018 05:25:10 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::544b:a68d:e6a5:ba6e]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::544b:a68d:e6a5:ba6e%2]) with mapi id 15.20.1294.044; Mon, 12 Nov 2018 05:25:09 +0000 From: Slava Ovsiienko To: Yongseok Koh CC: Shahaf Shuler , "dev@dpdk.org" Thread-Topic: [PATCH 3/3] net/mlx5: fix rule cleanup Netlink command sending Thread-Index: AQHUeNwQ2EdvUZRzrESa8460Z/OZZaVKdUSAgAEncNA= Date: Mon, 12 Nov 2018 05:25:09 +0000 Message-ID: References: <1541843951-31708-1-git-send-email-viacheslavo@mellanox.com> <1541843951-31708-4-git-send-email-viacheslavo@mellanox.com> <9DE1F3C9-D4C0-45B7-9B8C-DA9D80C53FC2@mellanox.com> In-Reply-To: <9DE1F3C9-D4C0-45B7-9B8C-DA9D80C53FC2@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR05MB3460; 6:diJ4Wtr8itunLAiKaCX1K2+U/QVWLrkTA4Ja2LyfA8DwM33RCmfi28s8piSRR3xKgf8+EJwTZCO2M4eSHxlvZCn5Oz3DVmTdmrqIz314n2PGidWr+Y5ad+gqc19RBux2n8ogMxbVZkhxQ8unm8i2dkU7xd0aJygWIJwkyywTKRztnO+z60xsznJOzn15Z8icncT0tokh8CZZ6rm4Ow9PG/o77HANyIr18iDvMW/vKkJf5gmMWV8o14oNbiRXfuLCqNYjzUdA/rygMSTy27Mh6D8ee4Lnhy9sW77wc57lYXkhx4Xnyf7wToFYjmJw5l8twogc5B3fLi6ZuRnYxMjC/Cww5aUZbOFgXw6V/pyO0no7NiJs1QLDT6/TNvNxRlTW0IK/+8hHB8rqi6RK9anUhJeJI7rCDRuhSxbUArFaKvGp4cCKceLGRO5mR5z3KVSZE42m7oa+09zS0qxgK7wBtQ==; 5:sGKnTEo3yCCDkUwvkaHiBPy6dqXHga40PujsVINulHW5JeOozG0yK+twKCWVHm3WYxivOMpe/ZObQjfvEYam2e/ZfdrPl68qJ5X+WxA/Shyg2DYNJxIiLRGmuZVlSelhGilvrb75kVWnl4ouB4CjBaCua6yByUtbbdTqKheBRbQ=; 7:wCXRPVMen5oWm4kMuq0EUyIT/g+djAm9sHeN5l1jXzQLvexTInzaffhu5kQE3insarDHLFJJxJ8cwIPGQjcMIYk9+fTNsFnCCJgoAelPcYki+clfOm9RzWcvfq2bej8DySwTVNBmqgeYO1btzucBPA== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: f9b42923-2d9e-49a8-eb1b-08d6485f36e3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390040)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR05MB3460; x-ms-traffictypediagnostic: AM4PR05MB3460: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3231382)(944501410)(52105112)(93006095)(93001095)(3002001)(6055026)(148016)(149066)(150057)(6041310)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(201708071742011)(7699051)(76991095); SRVR:AM4PR05MB3460; BCL:0; PCL:0; RULEID:; SRVR:AM4PR05MB3460; x-forefront-prvs: 0854128AF0 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(346002)(136003)(366004)(376002)(396003)(189003)(13464003)(199004)(53546011)(6506007)(66066001)(102836004)(76176011)(7696005)(186003)(99286004)(26005)(476003)(486006)(446003)(86362001)(105586002)(229853002)(11346002)(256004)(106356001)(14444005)(5660300001)(71190400001)(71200400001)(6436002)(8676002)(81166006)(81156014)(8936002)(54906003)(74316002)(68736007)(6116002)(3846002)(33656002)(6246003)(2900100001)(9686003)(53936002)(4326008)(305945005)(6862004)(55016002)(97736004)(7736002)(6636002)(478600001)(316002)(14454004)(2906002)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3460; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 5nRS8cSFE/KvD73x5lAfIJ5vwx37xFnsj/sH1ozuXy5deKJQ5Q4cIr5A7WOnMAqEDt+3yofQXYCAwqfhPl6jhDkhD3HAxyZionfh/AyIR2PuDCnOy4kE9wgA4i920lOh++h4IXisrraLPuX1wDkePLLT+BN9V5I0JjckAckdFig7CqJB9T4laPhhbI+7yfmYBdzMBc/hvI9pyVgh9DwpMpeFZ1B6frPTEQgZsOO1t91wt9CthiPlF66/cwheXumY4jhGC1GHZYJ+TiCOzjK7lBWSewnzEHk9/VMLWGXL8eQPsWp8n42FzXwGqxXs0lN/cY2jm23wHVG1BnlJ830+hc6JsfNhts64V/cF5i3pKgo= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: f9b42923-2d9e-49a8-eb1b-08d6485f36e3 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Nov 2018 05:25:09.8472 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3460 Subject: Re: [dpdk-dev] [PATCH 3/3] net/mlx5: fix rule cleanup Netlink command sending X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Nov 2018 05:25:18 -0000 > -----Original Message----- > From: Yongseok Koh > Sent: Sunday, November 11, 2018 13:42 > To: Slava Ovsiienko > Cc: Shahaf Shuler ; dev@dpdk.org > Subject: Re: [PATCH 3/3] net/mlx5: fix rule cleanup Netlink command sendi= ng >=20 >=20 > > On Nov 10, 2018, at 1:59 AM, Slava Ovsiienko > wrote: > > > > The VXLAN related rule cleanup routine queries and gathers all > > existing local IP and neigh rules into buffer list. One buffer may > > contain multiple rule deletetion commands and is prepared to send into > > Netlink as single message. But, if error occurs for some deletion > > commands in the buffer, the multiple ACK message with errors can be > > send back by the kernel. It breaks the Netlink communication sequence > > numbers, because we expect only one ACK message and it smashes out > > futher Netlik communication. >=20 > Just curious. > Is parsing the multiple ack msgs more complex than sending commands one > by one? We are in the midst of send query/get dump process. We can't send another request and wait ack for it - we are receiving the dump. Possible, it can be done via creation one more Netlink socket, but I'm not sure the requests are not queued by kernel. So - the simplest way - gather dump and then send commands. PS. Actually I have refactored gathering/sending, we need to gather parameters only, not build entire commands in callbacks, but this patch is not tested yet and too large as for simple fix.=20 WBR, Slava >=20 > > The workaround of this problem is to send rule deletion commands from > > buffer in one-by-one fashion and get ACK message for every command > > sent. We do not expect too may rules preexist, so there should not be > > critical performance degradation at VXLAN outer interface > > initialization. > > > > Fixes: f420f03d6772 ("net/mlx5: add E-switch VXLAN rule cleanup > > routines") > > > > Signed-off-by: Viacheslav Ovsiienko > > --- >=20 > Acked-by: Yongseok Koh >=20 > Thanks >=20 > > drivers/net/mlx5/mlx5_flow_tcf.c | 58 > > +++++++++++++++++----------------------- > > 1 file changed, 24 insertions(+), 34 deletions(-) > > > > diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c > > b/drivers/net/mlx5/mlx5_flow_tcf.c > > index bba8aed..21eb99e 100644 > > --- a/drivers/net/mlx5/mlx5_flow_tcf.c > > +++ b/drivers/net/mlx5/mlx5_flow_tcf.c > > @@ -3847,30 +3847,6 @@ struct tcf_nlcb_context { } > > > > /** > > - * Set NLM_F_ACK flags in the last netlink command in buffer. > > - * Only last command in the buffer will be acked by system. > > - * > > - * @param[in, out] buf > > - * Pointer to buffer with netlink commands. > > - */ > > -static void > > -flow_tcf_setack_nlcmd(struct tcf_nlcb_buf *buf) -{ > > - struct nlmsghdr *nlh; > > - uint32_t size =3D 0; > > - > > - assert(buf->size); > > - do { > > - nlh =3D (struct nlmsghdr *)&buf->msg[size]; > > - size +=3D NLMSG_ALIGN(nlh->nlmsg_len); > > - if (size >=3D buf->size) { > > - nlh->nlmsg_flags |=3D NLM_F_ACK; > > - break; > > - } > > - } while (true); > > -} > > - > > -/** > > * Send the buffers with prepared netlink commands. Scans the list and > > * sends all found buffers. Buffers are sent and freed anyway in order > > * to prevent memory leakage if some every message in received packet. > > @@ -3888,21 +3864,35 @@ struct tcf_nlcb_context { > > flow_tcf_send_nlcmd(struct mlx5_flow_tcf_context *tcf, > > struct tcf_nlcb_context *ctx) > > { > > - struct tcf_nlcb_buf *bc, *bn; > > - struct nlmsghdr *nlh; > > + struct tcf_nlcb_buf *bc =3D LIST_FIRST(&ctx->nlbuf); > > int ret =3D 0; > > > > - bc =3D LIST_FIRST(&ctx->nlbuf); > > while (bc) { > > + struct tcf_nlcb_buf *bn =3D LIST_NEXT(bc, next); > > + struct nlmsghdr *nlh; > > + uint32_t msg =3D 0; > > int rc; > > > > - bn =3D LIST_NEXT(bc, next); > > - if (bc->size) { > > - flow_tcf_setack_nlcmd(bc); > > - nlh =3D (struct nlmsghdr *)&bc->msg; > > - rc =3D flow_tcf_nl_ack(tcf, nlh, bc->size, NULL, NULL); > > - if (rc && !ret) > > - ret =3D rc; > > + while (msg < bc->size) { > > + /* > > + * Send Netlink commands from buffer in one by one > > + * fashion. If we send multiple rule deletion > commands > > + * in one Netlink message and some error occurs it > may > > + * cause multiple ACK error messages and break > sequence > > + * numbers of Netlink communication, because we > expect > > + * the only one ACK reply. > > + */ > > + assert((bc->size - msg) >=3D sizeof(struct nlmsghdr)); > > + nlh =3D (struct nlmsghdr *)&bc->msg[msg]; > > + assert((bc->size - msg) >=3D nlh->nlmsg_len); > > + msg +=3D nlh->nlmsg_len; > > + rc =3D flow_tcf_nl_ack(tcf, nlh, 0, NULL, NULL); > > + if (rc) { > > + DRV_LOG(WARNING, > > + "netlink: cleanup error %d", rc); > > + if (!ret) > > + ret =3D rc; > > + } > > } > > rte_free(bc); > > bc =3D bn; > > -- > > 1.8.3.1 > >