From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0071.outbound.protection.outlook.com [104.47.0.71]) by dpdk.org (Postfix) with ESMTP id D0D3F1B3B9 for ; Fri, 2 Nov 2018 00:59:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nSTGA4HeFwb5MJiaHGr/AGn1EiXl46ddL2NE5GF2cFs=; b=jRJtF11KkaYEwYXD+wTx8yzlX8YKF+CtIM+buLfNN+a6IViIu7S7WBOzD5PsXLm6+2fMrBavfBuY4iREZYrRNBx/o8+NmHZbb74OmwsAcdK8mv5x9C+/EsDpqp+ceWNgmNjYaiX+suGDc6Kr35DqgThU9YxNkDPJ0jaF+N4x3bs= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB4057.eurprd05.prod.outlook.com (52.134.67.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1273.21; Thu, 1 Nov 2018 23:59:49 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::f8a1:fcab:94f0:97cc]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::f8a1:fcab:94f0:97cc%4]) with mapi id 15.20.1273.030; Thu, 1 Nov 2018 23:59:49 +0000 From: Yongseok Koh To: Slava Ovsiienko CC: Shahaf Shuler , "dev@dpdk.org" Thread-Topic: [PATCH v3 11/13] net/mlx5: add e-switch VXLAN tunnel devices management Thread-Index: AQHUcd0m1hNdeMmESE+Fv36axbjHyKU7miwA Date: Thu, 1 Nov 2018 23:59:49 +0000 Message-ID: <20181101235939.GL6118@mtidpdk.mti.labs.mlnx> References: <1539612815-47199-1-git-send-email-viacheslavo@mellanox.com> <1541074741-41368-1-git-send-email-viacheslavo@mellanox.com> <1541074741-41368-12-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1541074741-41368-12-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR11CA0053.namprd11.prod.outlook.com (2603:10b6:a03:80::30) To DB3PR0502MB3980.eurprd05.prod.outlook.com (2603:10a6:8:10::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [209.116.155.178] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB4057; 6:hKEWHKkIPux5kpuKvZbPe52FxbBHXt5UEI9FIzvx4YTw9rPpX8zhJgEQ9hjttu1dcogpT9aSQ2wfCGD4oevz5wyzr14hyaSeHnWnIYiy3ir6U1ujt7kxEzeTGD6gUOXczU18LoCcwpqnQYhW3s6I88+iapi/RG/EHmNOi+W4K5l5RqpkQdrmqK6fLrXDjIXtVse1zZUaVJ3MjoAWfq0MHTg4yNsYMEXjDnbi2hnVHif8YO1XfJznmkVIg68ZdZ3pgcVxfQPzuQDIK9H7Ndg12MGkK4ehYFIUZI3ZbyJpc1QzuiBD+pk/JHClYJlvuSdzTfDTikTw/sC+i/LdbYudlgvY55A/j5R5jxsykUya1oyhstUdLpkK3K8YBHbo0o2bsrcqLVqxCPImU6Z3fkCCVZAXzIb9BM6C31S2zDSp88oN6OOgwMge9yrKDwwxeioyewMPNfrY39EsYIz5LZbyhA==; 5:S9TSIgpC7D2iVPFg/nmweFlgKZV1sUlH7fAFb59Ln/enmLxTkkarAuI9uv3TEmGeYdKolLHRPWTo+a0V+RYN6MIR0vKNHQegY7Zrbj06Ikfm/BAbxjL34MdaokOiah+lYsWaHAjEV/frvUBJVXQqcvCF8vniv3AQnBrbEniPRzA=; 7:gTlwHxJWZjhI6UvdWFLmP8vbwkB+bxEaW9t/6e+mY1qhVxJy+n93pK6B5l37xfBap5G0571U6y51Q8zWfhjayl4pSKal9BMSxvob/ZxZltWwYpl5yh0lMdbt7fKtcsc73k3pNwedehsFsHbcZLavZg== x-ms-office365-filtering-correlation-id: fcfa3ab4-e38e-4d6f-8a48-08d640561ad3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB4057; x-ms-traffictypediagnostic: DB3PR0502MB4057: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(211171220733660)(788757137089); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231382)(944501410)(52105095)(6055026)(148016)(149066)(150057)(6041310)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB3PR0502MB4057; BCL:0; PCL:0; RULEID:; SRVR:DB3PR0502MB4057; x-forefront-prvs: 0843C17679 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(396003)(366004)(136003)(39860400002)(199004)(189003)(76176011)(1076002)(478600001)(105586002)(99286004)(33656002)(14454004)(102836004)(106356001)(26005)(11346002)(186003)(476003)(386003)(6246003)(4326008)(6512007)(33896004)(486006)(6506007)(446003)(52116002)(6636002)(9686003)(68736007)(2906002)(6436002)(53946003)(97736004)(25786009)(2900100001)(53936002)(305945005)(7736002)(86362001)(81156014)(71200400001)(6486002)(229853002)(66066001)(5024004)(6116002)(3846002)(8676002)(316002)(256004)(14444005)(6862004)(81166006)(5660300001)(8936002)(54906003)(4744004)(71190400001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4057; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: zZqsUECVSfj7IajJ2y79y+uuXLJ6Jj3WhtjDM8ynrXxYYUyT0EvpO6l+xNdBZajZWIDPNKD7shclcWHJwLetxITegyODiUuFUXvBxWi7I+yCrfvCRMhJze1tBnqm8aQdpBH1FhAnXAg32KSLhEQpdvxJ7AH3PqM1MYmZeWe11gh6GF1dUiYc2BnK3kiUF0n9IL8ZvLZrKJIpQl3NcpgKB01iJ953OXpz8IqjM0fsLjT1exZkrTCEnYyO5LIjyJuHVz7tjMNENVUxL6SIi/cwMViZzfhVi0rFFcq13JnZ9UllyWwLwkjOP/wAgRcVSQmnhzXU8ToQiLiUZjZxT/eoEjyMi9iOKEXb8U1Y8PMUPiA= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: <630A068C39C97849ACC0CA8042432F54@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: fcfa3ab4-e38e-4d6f-8a48-08d640561ad3 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2018 23:59:49.2498 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4057 Subject: Re: [dpdk-dev] [PATCH v3 11/13] net/mlx5: add e-switch VXLAN tunnel devices management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 23:59:52 -0000 On Thu, Nov 01, 2018 at 05:19:33AM -0700, Slava Ovsiienko wrote: > VXLAN interfaces are dynamically created for each local UDP port > of outer networks and then used as targets for TC "flower" filters > in order to perform encapsulation. These VXLAN interfaces are > system-wide, the only one device with given UDP port can exist > in the system (the attempt of creating another device with the > same UDP local port returns EEXIST), so PMD should support the > shared device instances database for PMD instances. These VXLAN > implicitly created devices are called VTEPs (Virtual Tunnel > End Points). >=20 > Creation of the VTEP occurs at the moment of rule applying. The > link is set up, root ingress qdisc is also initialized. >=20 > Encapsulation VTEPs are created on per port basis, the single > VTEP is attached to the outer interface and is shared for all > encapsulation rules on this interface. The source UDP port is > automatically selected in range 30000-60000. >=20 > For decapsulaton one VTEP is created per every unique UDP > local port to accept tunnel traffic. The name of created > VTEP consists of prefix "vmlx_" and the number of UDP port in > decimal digits without leading zeros (vmlx_4789). The VTEP > can be preliminary created in the system before the launching > application, it allows to share UDP ports between primary > and secondary processes. >=20 > Suggested-by: Adrien Mazarguil > Signed-off-by: Viacheslav Ovsiienko > --- Acked-by: Yongseok Koh Thanks > drivers/net/mlx5/mlx5_flow_tcf.c | 461 +++++++++++++++++++++++++++++++++= +++++- > 1 file changed, 455 insertions(+), 6 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c b/drivers/net/mlx5/mlx5_flo= w_tcf.c > index 02b64d6..c6e07f5 100644 > --- a/drivers/net/mlx5/mlx5_flow_tcf.c > +++ b/drivers/net/mlx5/mlx5_flow_tcf.c > @@ -3750,6 +3750,417 @@ struct pedit_parser { > return -err; > } > =20 > +#define MNL_BUF_EXTRA_SPACE 16 > +#define MNL_REQUEST_SIZE_MIN 256 > +#define MNL_REQUEST_SIZE_MAX 2048 > +#define MNL_REQUEST_SIZE RTE_MIN(RTE_MAX(sysconf(_SC_PAGESIZE), \ > + MNL_REQUEST_SIZE_MIN), MNL_REQUEST_SIZE_MAX) > + > +/* VTEP device list is shared between PMD port instances. */ > +static LIST_HEAD(, tcf_vtep) vtep_list_vxlan =3D LIST_HEAD_INITIALIZER()= ; > +static pthread_mutex_t vtep_list_mutex =3D PTHREAD_MUTEX_INITIALIZER; > + > +/** > + * Deletes VTEP network device. > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] vtep > + * Object represinting the network device to delete. Memory > + * allocated for this object is freed by routine. > + */ > +static void > +flow_tcf_vtep_delete(struct mlx5_flow_tcf_context *tcf, > + struct tcf_vtep *vtep) > +{ > + struct nlmsghdr *nlh; > + struct ifinfomsg *ifm; > + alignas(struct nlmsghdr) > + uint8_t buf[mnl_nlmsg_size(MNL_ALIGN(sizeof(*ifm))) + > + MNL_BUF_EXTRA_SPACE]; > + int ret; > + > + assert(!vtep->refcnt); > + /* Delete only ifaces those we actually created. */ > + if (vtep->created && vtep->ifindex) { > + DRV_LOG(INFO, "VTEP delete (%d)", vtep->ifindex); > + nlh =3D mnl_nlmsg_put_header(buf); > + nlh->nlmsg_type =3D RTM_DELLINK; > + nlh->nlmsg_flags =3D NLM_F_REQUEST; > + ifm =3D mnl_nlmsg_put_extra_header(nlh, sizeof(*ifm)); > + ifm->ifi_family =3D AF_UNSPEC; > + ifm->ifi_index =3D vtep->ifindex; > + assert(sizeof(buf) >=3D nlh->nlmsg_len); > + ret =3D flow_tcf_nl_ack(tcf, nlh, 0, NULL, NULL); > + if (ret) > + DRV_LOG(WARNING, "netlink: error deleting vxlan" > + " encap/decap ifindex %u", > + ifm->ifi_index); > + } > + rte_free(vtep); > +} > + > +/** > + * Creates VTEP network device. > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] ifouter > + * Outer interface to attach new-created VXLAN device > + * If zero the VXLAN device will not be attached to any device. > + * These VTEPs are used for decapsulation and can be precreated > + * and shared between processes. > + * @param[in] port > + * UDP port of created VTEP device. > + * @param[out] error > + * Perform verbose error reporting if not NULL. > + * > + * @return > + * Pointer to created device structure on success, > + * NULL otherwise and rte_errno is set. > + */ > +#ifdef HAVE_IFLA_VXLAN_COLLECT_METADATA > +static struct tcf_vtep* > +flow_tcf_vtep_create(struct mlx5_flow_tcf_context *tcf, > + unsigned int ifouter, > + uint16_t port, struct rte_flow_error *error) > +{ > + struct tcf_vtep *vtep; > + struct nlmsghdr *nlh; > + struct ifinfomsg *ifm; > + char name[sizeof(MLX5_VXLAN_DEVICE_PFX) + 24]; > + alignas(struct nlmsghdr) > + uint8_t buf[mnl_nlmsg_size(sizeof(*ifm)) + > + SZ_NLATTR_DATA_OF(sizeof(name)) + > + SZ_NLATTR_NEST * 2 + > + SZ_NLATTR_STRZ_OF("vxlan") + > + SZ_NLATTR_DATA_OF(sizeof(uint32_t)) + > + SZ_NLATTR_DATA_OF(sizeof(uint16_t)) + > + SZ_NLATTR_DATA_OF(sizeof(uint8_t)) * 3 + > + MNL_BUF_EXTRA_SPACE]; > + struct nlattr *na_info; > + struct nlattr *na_vxlan; > + rte_be16_t vxlan_port =3D rte_cpu_to_be_16(port); > + int ret; > + > + vtep =3D rte_zmalloc(__func__, sizeof(*vtep), alignof(struct tcf_vtep))= ; > + if (!vtep) { > + rte_flow_error_set > + (error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, "unadble to allocate memory for VTEP desc"); > + return NULL; > + } > + *vtep =3D (struct tcf_vtep){ > + .port =3D port, > + .local =3D LIST_HEAD_INITIALIZER(), > + .neigh =3D LIST_HEAD_INITIALIZER(), > + }; > + memset(buf, 0, sizeof(buf)); > + nlh =3D mnl_nlmsg_put_header(buf); > + nlh->nlmsg_type =3D RTM_NEWLINK; > + nlh->nlmsg_flags =3D NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; > + ifm =3D mnl_nlmsg_put_extra_header(nlh, sizeof(*ifm)); > + ifm->ifi_family =3D AF_UNSPEC; > + ifm->ifi_type =3D 0; > + ifm->ifi_index =3D 0; > + ifm->ifi_flags =3D IFF_UP; > + ifm->ifi_change =3D 0xffffffff; > + snprintf(name, sizeof(name), "%s%u", MLX5_VXLAN_DEVICE_PFX, port); > + mnl_attr_put_strz(nlh, IFLA_IFNAME, name); > + na_info =3D mnl_attr_nest_start(nlh, IFLA_LINKINFO); > + assert(na_info); > + mnl_attr_put_strz(nlh, IFLA_INFO_KIND, "vxlan"); > + na_vxlan =3D mnl_attr_nest_start(nlh, IFLA_INFO_DATA); > + if (ifouter) > + mnl_attr_put_u32(nlh, IFLA_VXLAN_LINK, ifouter); > + assert(na_vxlan); > + mnl_attr_put_u8(nlh, IFLA_VXLAN_COLLECT_METADATA, 1); > + mnl_attr_put_u8(nlh, IFLA_VXLAN_UDP_ZERO_CSUM6_RX, 1); > + mnl_attr_put_u8(nlh, IFLA_VXLAN_LEARNING, 0); > + mnl_attr_put_u16(nlh, IFLA_VXLAN_PORT, vxlan_port); > + mnl_attr_nest_end(nlh, na_vxlan); > + mnl_attr_nest_end(nlh, na_info); > + assert(sizeof(buf) >=3D nlh->nlmsg_len); > + ret =3D flow_tcf_nl_ack(tcf, nlh, 0, NULL, NULL); > + if (ret) { > + DRV_LOG(WARNING, > + "netlink: VTEP %s create failure (%d)", > + name, rte_errno); > + if (rte_errno !=3D EEXIST || ifouter) > + /* > + * Some unhandled error occurred or device is > + * for encapsulation and cannot be shared. > + */ > + goto error; > + } else { > + /* > + * Mark device we actually created. > + * We should explicitly delete > + * when we do not need it anymore. > + */ > + vtep->created =3D 1; > + } > + /* Try to get ifindex of created of pre-existing device. */ > + ret =3D if_nametoindex(name); > + if (!ret) { > + DRV_LOG(WARNING, > + "VTEP %s failed to get index (%d)", name, errno); > + rte_flow_error_set > + (error, -errno, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "netlink: failed to retrieve VTEP ifindex"); > + goto error; > + } > + vtep->ifindex =3D ret; > + vtep->ifouter =3D ifouter; > + memset(buf, 0, sizeof(buf)); > + nlh =3D mnl_nlmsg_put_header(buf); > + nlh->nlmsg_type =3D RTM_NEWLINK; > + nlh->nlmsg_flags =3D NLM_F_REQUEST; > + ifm =3D mnl_nlmsg_put_extra_header(nlh, sizeof(*ifm)); > + ifm->ifi_family =3D AF_UNSPEC; > + ifm->ifi_type =3D 0; > + ifm->ifi_index =3D vtep->ifindex; > + ifm->ifi_flags =3D IFF_UP; > + ifm->ifi_change =3D IFF_UP; > + ret =3D flow_tcf_nl_ack(tcf, nlh, 0, NULL, NULL); > + if (ret) { > + rte_flow_error_set(error, -errno, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "netlink: failed to set VTEP link up"); > + DRV_LOG(WARNING, "netlink: VTEP %s set link up failure (%d)", > + name, rte_errno); > + goto clean; > + } > + ret =3D mlx5_flow_tcf_init(tcf, vtep->ifindex, error); > + if (ret) { > + DRV_LOG(WARNING, "VTEP %s init failure (%d)", name, rte_errno); > + goto clean; > + } > + DRV_LOG(INFO, "VTEP create (%d, %d)", vtep->port, vtep->ifindex); > + vtep->refcnt =3D 1; > + return vtep; > +clean: > + flow_tcf_vtep_delete(tcf, vtep); > + return NULL; > +error: > + rte_free(vtep); > + return NULL; > +} > +#else > +static struct tcf_vtep* > +flow_tcf_vtep_create(struct mlx5_flow_tcf_context *tcf __rte_unused, > + unsigned int ifouter __rte_unused, > + uint16_t port __rte_unused, > + struct rte_flow_error *error) > +{ > + rte_flow_error_set(error, ENOTSUP, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "netlink: failed to create VTEP, " > + "VXLAN metadata are not supported by kernel"); > + return NULL; > +} > +#endif /* HAVE_IFLA_VXLAN_COLLECT_METADATA */ > + > +/** > + * Acquire target interface index for VXLAN tunneling decapsulation. > + * In order to share the UDP port within the other interfaces the > + * VXLAN device created as not attached to any interface (if created). > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] dev_flow > + * Flow tcf object with tunnel structure pointer set. > + * @param[out] error > + * Perform verbose error reporting if not NULL. > + * @return > + * Interface descriptor pointer on success, > + * NULL otherwise and rte_errno is set. > + */ > +static struct tcf_vtep* > +flow_tcf_decap_vtep_acquire(struct mlx5_flow_tcf_context *tcf, > + struct mlx5_flow *dev_flow, > + struct rte_flow_error *error) > +{ > + struct tcf_vtep *vtep; > + uint16_t port =3D dev_flow->tcf.vxlan_decap->udp_port; > + > + LIST_FOREACH(vtep, &vtep_list_vxlan, next) { > + if (vtep->port =3D=3D port) > + break; > + } > + if (vtep && vtep->ifouter) { > + rte_flow_error_set(error, -errno, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "Failed to create decap VTEP with specified" > + " UDP port, atatched device exists"); > + return NULL; > + } > + if (vtep) { > + /* Device exists, just increment the reference counter. */ > + vtep->refcnt++; > + assert(vtep->ifindex); > + return vtep; > + } > + /* No decapsulation device exists, try to create the new one. */ > + vtep =3D flow_tcf_vtep_create(tcf, 0, port, error); > + if (vtep) > + LIST_INSERT_HEAD(&vtep_list_vxlan, vtep, next); > + return vtep; > +} > + > +/** > + * Aqcuire target interface index for VXLAN tunneling encapsulation. > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] ifouter > + * Network interface index to attach VXLAN encap device to. > + * @param[in] dev_flow > + * Flow tcf object with tunnel structure pointer set. > + * @param[out] error > + * Perform verbose error reporting if not NULL. > + * @return > + * Interface descriptor pointer on success, > + * NULL otherwise and rte_errno is set. > + */ > +static struct tcf_vtep* > +flow_tcf_encap_vtep_acquire(struct mlx5_flow_tcf_context *tcf, > + unsigned int ifouter, > + struct mlx5_flow *dev_flow __rte_unused, > + struct rte_flow_error *error) > +{ > + static uint16_t encap_port =3D MLX5_VXLAN_PORT_MIN - 1; > + struct tcf_vtep *vtep; > + > + assert(ifouter); > + /* Look whether the attached VTEP for encap is created. */ > + LIST_FOREACH(vtep, &vtep_list_vxlan, next) { > + if (vtep->ifouter =3D=3D ifouter) > + break; > + } > + if (vtep) { > + /* VTEP already exists, just increment the reference. */ > + vtep->refcnt++; > + } else { > + uint16_t pcnt; > + > + /* Not found, we should create the new attached VTEP. */ > + for (pcnt =3D 0; pcnt <=3D (MLX5_VXLAN_PORT_MAX > + - MLX5_VXLAN_PORT_MIN); pcnt++) { > + encap_port++; > + /* Wraparound the UDP port index. */ > + if (encap_port < MLX5_VXLAN_PORT_MIN || > + encap_port > MLX5_VXLAN_PORT_MAX) > + encap_port =3D MLX5_VXLAN_PORT_MIN; > + /* Check whether UDP port is in already in use. */ > + LIST_FOREACH(vtep, &vtep_list_vxlan, next) { > + if (vtep->port =3D=3D encap_port) > + break; > + } > + if (vtep) { > + /* Port is in use, try the next one. */ > + vtep =3D NULL; > + continue; > + } > + vtep =3D flow_tcf_vtep_create(tcf, ifouter, > + encap_port, error); > + if (vtep) { > + LIST_INSERT_HEAD(&vtep_list_vxlan, vtep, next); > + break; > + } > + if (rte_errno !=3D EEXIST) > + break; > + } > + if (!vtep) > + return NULL; > + } > + assert(vtep->ifouter =3D=3D ifouter); > + assert(vtep->ifindex); > + return vtep; > +} > + > +/** > + * Acquires target interface index for tunneling of any type. > + * Creates the new VTEP if needed. > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] ifouter > + * Network interface index to attach VXLAN encap device to. > + * @param[in] dev_flow > + * Flow tcf object with tunnel structure pointer set. > + * @param[out] error > + * Perform verbose error reporting if not NULL. > + * @return > + * Interface descriptor pointer on success, > + * NULL otherwise and rte_errno is set. > + */ > +static struct tcf_vtep* > +flow_tcf_vtep_acquire(struct mlx5_flow_tcf_context *tcf, > + unsigned int ifouter, > + struct mlx5_flow *dev_flow, > + struct rte_flow_error *error) > +{ > + struct tcf_vtep *vtep =3D NULL; > + > + assert(dev_flow->tcf.tunnel); > + pthread_mutex_lock(&vtep_list_mutex); > + switch (dev_flow->tcf.tunnel->type) { > + case FLOW_TCF_TUNACT_VXLAN_ENCAP: > + vtep =3D flow_tcf_encap_vtep_acquire(tcf, ifouter, > + dev_flow, error); > + break; > + case FLOW_TCF_TUNACT_VXLAN_DECAP: > + vtep =3D flow_tcf_decap_vtep_acquire(tcf, dev_flow, error); > + break; > + default: > + rte_flow_error_set(error, ENOTSUP, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "unsupported tunnel type"); > + break; > + } > + pthread_mutex_unlock(&vtep_list_mutex); > + return vtep; > +} > + > +/** > + * Release tunneling interface by ifindex. Decrements reference > + * counter and actually removes the device if counter is zero. > + * > + * @param[in] tcf > + * Context object initialized by mlx5_flow_tcf_context_create(). > + * @param[in] vtep > + * VTEP device descriptor structure. > + * @param[in] dev_flow > + * Flow tcf object with tunnel structure pointer set. > + */ > +static void > +flow_tcf_vtep_release(struct mlx5_flow_tcf_context *tcf, > + struct tcf_vtep *vtep, > + struct mlx5_flow *dev_flow) > +{ > + assert(dev_flow->tcf.tunnel); > + pthread_mutex_lock(&vtep_list_mutex); > + switch (dev_flow->tcf.tunnel->type) { > + case FLOW_TCF_TUNACT_VXLAN_DECAP: > + break; > + case FLOW_TCF_TUNACT_VXLAN_ENCAP: > + break; > + default: > + assert(false); > + DRV_LOG(WARNING, "Unsupported tunnel type"); > + break; > + } > + assert(vtep->refcnt); > + if (--vtep->refcnt =3D=3D 0) { > + LIST_REMOVE(vtep, next); > + flow_tcf_vtep_delete(tcf, vtep); > + } > + pthread_mutex_unlock(&vtep_list_mutex); > +} > + > + > /** > * Apply flow to E-Switch by sending Netlink message. > * > @@ -3775,11 +4186,35 @@ struct pedit_parser { > dev_flow =3D LIST_FIRST(&flow->dev_flows); > /* E-Switch flow can't be expanded. */ > assert(!LIST_NEXT(dev_flow, next)); > + if (dev_flow->tcf.applied) > + return 0; > nlh =3D dev_flow->tcf.nlh; > nlh->nlmsg_type =3D RTM_NEWTFILTER; > nlh->nlmsg_flags =3D NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; > - if (!flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL)) > + if (dev_flow->tcf.tunnel) { > + /* > + * Replace the interface index, target for > + * encapsulation, source for decapsulation. > + */ > + assert(!dev_flow->tcf.tunnel->vtep); > + assert(dev_flow->tcf.tunnel->ifindex_ptr); > + /* Acquire actual VTEP device when rule is being applied. */ > + dev_flow->tcf.tunnel->vtep =3D > + flow_tcf_vtep_acquire(ctx, > + dev_flow->tcf.tunnel->ifindex_org, > + dev_flow, error); > + if (!dev_flow->tcf.tunnel->vtep) > + return -rte_errno; > + DRV_LOG(INFO, "Replace ifindex: %d->%d", > + dev_flow->tcf.tunnel->vtep->ifindex, > + dev_flow->tcf.tunnel->ifindex_org); > + *dev_flow->tcf.tunnel->ifindex_ptr =3D > + dev_flow->tcf.tunnel->vtep->ifindex; > + } > + if (!flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL)) { > + dev_flow->tcf.applied =3D 1; > return 0; > + } > return rte_flow_error_set(error, rte_errno, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > "netlink: failed to create TC flow rule"); > @@ -3808,10 +4243,20 @@ struct pedit_parser { > return; > /* E-Switch flow can't be expanded. */ > assert(!LIST_NEXT(dev_flow, next)); > - nlh =3D dev_flow->tcf.nlh; > - nlh->nlmsg_type =3D RTM_DELTFILTER; > - nlh->nlmsg_flags =3D NLM_F_REQUEST; > - flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL); > + if (dev_flow->tcf.applied) { > + nlh =3D dev_flow->tcf.nlh; > + nlh->nlmsg_type =3D RTM_DELTFILTER; > + nlh->nlmsg_flags =3D NLM_F_REQUEST; > + flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL); > + if (dev_flow->tcf.tunnel) { > + assert(dev_flow->tcf.tunnel->vtep); > + flow_tcf_vtep_release(ctx, > + dev_flow->tcf.tunnel->vtep, > + dev_flow); > + dev_flow->tcf.tunnel->vtep =3D NULL; > + } > + dev_flow->tcf.applied =3D 0; > + } > } > =20 > /** > @@ -4338,7 +4783,9 @@ struct pedit_parser { > struct nlmsghdr *nlh; > struct tcmsg *tcm; > alignas(struct nlmsghdr) > - uint8_t buf[mnl_nlmsg_size(sizeof(*tcm) + 128)]; > + uint8_t buf[mnl_nlmsg_size(sizeof(*tcm)) + > + SZ_NLATTR_STRZ_OF("ingress") + > + MNL_BUF_EXTRA_SPACE]; > =20 > /* Destroy existing ingress qdisc and everything attached to it. */ > nlh =3D mnl_nlmsg_put_header(buf); > @@ -4349,6 +4796,7 @@ struct pedit_parser { > tcm->tcm_ifindex =3D ifindex; > tcm->tcm_handle =3D TC_H_MAKE(TC_H_INGRESS, 0); > tcm->tcm_parent =3D TC_H_INGRESS; > + assert(sizeof(buf) >=3D nlh->nlmsg_len); > /* Ignore errors when qdisc is already absent. */ > if (flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL) && > rte_errno !=3D EINVAL && rte_errno !=3D ENOENT) > @@ -4366,6 +4814,7 @@ struct pedit_parser { > tcm->tcm_handle =3D TC_H_MAKE(TC_H_INGRESS, 0); > tcm->tcm_parent =3D TC_H_INGRESS; > mnl_attr_put_strz_check(nlh, sizeof(buf), TCA_KIND, "ingress"); > + assert(sizeof(buf) >=3D nlh->nlmsg_len); > if (flow_tcf_nl_ack(ctx, nlh, 0, NULL, NULL)) > return rte_flow_error_set(error, rte_errno, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > --=20 > 1.8.3.1 >=20