From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30086.outbound.protection.outlook.com [40.107.3.86]) by dpdk.org (Postfix) with ESMTP id 1BD5523D for ; Mon, 15 Oct 2018 16:14:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cTZ2fXdYgAQrE1eKO9+i4yjOEu/QqYZIMwIK8qo0f20=; b=WArClOdHkto69oWNTKNcY38WjgmIoSTfPWiNLbciXinsQBZoIkom8WmTktkRKOKUVWQUUbUO+BQmI7qN2UBtSgsyafuoPkr3Sl69/nXe9u4rVWUg/r4M35ed1ofH5tdf9hRYZ6MG7CRQ9m7hnrUXK2H7qNA6pDU3alRizdrBodY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; Received: from mellanox.com (37.142.13.130) by VI1PR05MB3277.eurprd05.prod.outlook.com (2603:10a6:802:1c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1228.24; Mon, 15 Oct 2018 14:14:08 +0000 From: Viacheslav Ovsiienko To: shahafs@mellanox.com, yskoh@mellanox.com Cc: dev@dpdk.org, Viacheslav Ovsiienko Date: Mon, 15 Oct 2018 14:13:28 +0000 Message-Id: <1539612815-47199-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1538461807-37507-1-git-send-email-viacheslavo@mellanox.com> References: <1538461807-37507-1-git-send-email-viacheslavo@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: CWLP265CA0084.GBRP265.PROD.OUTLOOK.COM (2603:10a6:401:50::24) To VI1PR05MB3277.eurprd05.prod.outlook.com (2603:10a6:802:1c::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 65955843-2b70-49c1-447f-08d632a87933 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR05MB3277; X-Microsoft-Exchange-Diagnostics: 1; VI1PR05MB3277; 3:Ah/LrweBybLzHUWL+ZQUSoQBTfuUwl3TDdRKC8mZxYnDW6nTIcJE8t90nHyP3pn72/jT/GyF99kGgAq66DUfdl77Jm4mHNKRyVtvrcIeinOstzHIXS5WjAIt3fbIeg9rPRK0Bczwgk44uspU0XuNxwLz7HRtyXQT/01hMDT05+AJ4quu2gf00MGeA+szLyzd3F/p3DcmFw/TEjXr91Y132znsXcHxGd7zeV41PlhAoK2pTeUT9YoDBeILBNjoHye; 25:LuAp/kT1aVPuG/pugrSkJqb6A5AHn8789LeYZL6sh+l5lC/ofEEYH7l+e5FRuUan9no+iOpcn8uCJYRZlBOWCcEsZYNRFMoDB/N3DaB8qPzObwNKlEU4aM+MDeJOyhV6Iso81h8mgPsUqX8rqqu2MwR1xAG3mNSLVvqBM2jid9PsAJYzxC9JbyOybng0leyai8Rs//Q7TksJl7bbIxh/EYJVNNDy674uhQkj4oVI8aDEW96KDN4QJrZL/xS2flSx7Kx3SxYcvI71fuk3be/l1sh9iLfev2+sxainK/2MuMWB08JLZHu3fwzbxRoD5TgoUtI6V4xECt2Z0+7/0FL2VA==; 31:YwgNkvinDVxQrHJGu5RfS6KU8GKGOV0gG0G56tdnaxgotqXckVwc12ZkW5JDMKykY4GALKu2loR8EQWENSRqD6l5rhB0GTti0U/0BKLFqxRxm03dWUvu5QgwnZZbIXCVaZGZTH5AQX5hPanl7+2Ex1DjMI9Cfs5OM7zzfhc8lRXj1+VTswwMtrOWcyYDDprfY0i5xGZM426yVcg/xRZdkZQpP5mdFhZYKX6H4Y5mB2A= X-MS-TrafficTypeDiagnostic: VI1PR05MB3277: X-Microsoft-Exchange-Diagnostics: 1; VI1PR05MB3277; 20:EB+sb84d5qzEjZMMZQNBFZwQ9K52TpEOHWfYHqijbi7EiZfCHNpI8S1NTn/v9mq8C6Z24gGZlo74gmNLZe3mOckdnEPdfQH3Ine1HekOw/v1qmeofowocONOt4Qhn6m+UFYgCMktMPFlrfLKzC2olSc9G0SRaANHkqDfqTxlj5Kn5awakSE2dcmzV3VjGlVRmWcOLLWPa+0ODdyDk14Y3tpZfSMKnpwD9n0s2tkGBg5ePFM57XIHtTytnVI6cuylEoDCvWvyvNy4uKJwHuUMUfhL8F1YPrrfkxLEmhvvH6jhnIk+1oFBeVVN15eYCMm6iGZakhK2vUWVskpebQkQu34t1PwMyToQXxy7w8SMBdwVE7ILJU1GwCN0g80Rc4SVV/K1QYJJMZGDuzeiIJAssfCfBIz1K5MOZE9Q0uyw2YvtqFl2K1Rg+NvlAQr1F9GUTREKRtGDRgzodWtIj+RkIL7G5SCe0BLRICvt1oCvPob0R5oL8RhjfFsKQYj5hMFB; 4:XQJ2Z0whSTpFYmG48ad0qP1fwCSFe6GSUWQaJjbJlRTFLlnnUQRzDGs/RrcOV3DeUA6KCexNRZ2VglXOwjkE6s2Oz+2NxjvOFx9+ceA1SOT+JJZutYuErJ9pmxnkwB0zAk7UrpTYdUiJGxa326CNCyPLiZUlEsvwR5IkMArnVtPOtMEfLs9lfpuOzQ8U9/nkEnVojSP5jguRTx1ixZNo7ttIJcFC+R0xMY5laErQFV0UDlF2VjUJ/HQlc4Hdes/8ueVkx0m7TtnO7gmV7Df1bw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(3231355)(944501410)(4982022)(52105095)(10201501046)(93006095)(93001095)(6055026)(149066)(150057)(6041310)(20161123560045)(20161123558120)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051); SRVR:VI1PR05MB3277; BCL:0; PCL:0; RULEID:; SRVR:VI1PR05MB3277; X-Forefront-PRVS: 0826B2F01B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(136003)(39860400002)(376002)(346002)(366004)(396003)(199004)(189003)(107886003)(16526019)(66066001)(7736002)(8676002)(47776003)(106356001)(51416003)(7696005)(52116002)(25786009)(68736007)(105586002)(186003)(16586007)(305945005)(8936002)(3846002)(81166006)(81156014)(6116002)(386003)(50226002)(6666004)(50466002)(11346002)(5024004)(86362001)(14444005)(446003)(76176011)(97736004)(6636002)(21086003)(36756003)(2906002)(33026002)(4720700003)(69596002)(6306002)(53936002)(55016002)(48376002)(85306007)(478600001)(476003)(8886007)(561944003)(316002)(26005)(966005)(4326008)(486006)(2616005)(5660300001)(956004); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR05MB3277; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR05MB3277; 23:kCZeOElbIGZ5fzEHix/jrvGmkI9gbiFoxc7r2+z1E?= =?us-ascii?Q?K36X5f7ALDGhD9LGIS3+ujFk9wCKZ1y6PRe3YNQwA7LeGTUa6XRzZ97o/61/?= =?us-ascii?Q?4sEGPZp4I/Mz/I7sqtbF0XylFDDmeGXH/PihK9TCw0C6B3MhgFmvMU4CyWZR?= =?us-ascii?Q?Vh0nzVttwkaH2njatNhWND/bistd4eEq7PpMVZDKsJ5F1UfJfRWR9ZWDL3rt?= =?us-ascii?Q?f32Vm5ou1yaoDZp50Dom6pf6bTjvMMwgQdDld1LH5Mga5AMdnbVeBTGJLbl3?= =?us-ascii?Q?55K1FDzjtM/NN0RjYAi2rO7YIKL4lcNIW3VMXg/xQt7s8qPzpeiOFTBsaugQ?= =?us-ascii?Q?PawSAAvjyp5rgK2goF3N3RQRDTaCMRiI+jgZka/vCqhN+4MUV2MYmywk2PKd?= =?us-ascii?Q?eb/90AxG5yq/mR8IRC0Q3i/ZIOfH8yXbxUATb7CM7SdwGMuE+YcyThBZFvgZ?= =?us-ascii?Q?0RMaK0mxKM5pZORHGtgyklcdCUNuOZyfaZuhemxbvBvZEukgNHyEOukkrFPq?= =?us-ascii?Q?fKtV/rCrmnTHt+/hRNriaZNT12VaLrAR30tMdTzkF0MShaCzw4nf/yRmplmH?= =?us-ascii?Q?kdlVzfhVavj40t60sWdVQBF5VoTakYQyKAcy6qUz876CJm0pWn4cY525ryLw?= =?us-ascii?Q?MQYsqop9cOQTv2ge7/FlLsOonJNAAhcJJ4V2ltUulPOY2tu66FdT0ixtawQE?= =?us-ascii?Q?e6tZj3ChlsRU6SLD+dfdzWDIyQ8RsE1DTNpbahm1AZ1rfNH2xi4g41XeZvql?= =?us-ascii?Q?FBqP6IRRHGQyYFuZZVsipxNxrgR7erxYF56Z1VvUniwvLdfR2bE7AcvqZI2Q?= =?us-ascii?Q?QXyQLxJ2NnwvGDXlnZ14oBG6MxlrQF0IBEN83tpo/Z75wb1gIscm5LpxSHvZ?= =?us-ascii?Q?2CFZj2Fr4+2qxrs+oWP5KvjSkWwiSax4twOfV8qY3a3AOKJwBpeYBgQNBUUJ?= =?us-ascii?Q?C+LO+Fsk8XsVNYQmglDAG75lLqsUZlNlOdG9cUSDrr7t1a7neuJPanabu1xT?= =?us-ascii?Q?gWMEbuIOCUscXCYYjqdoytIuzGjENPIXPLLHyIS9DR1rg1tAqjvhZcyJQACm?= =?us-ascii?Q?2M7b3SmUEoLGPRHozlAIE38pr1al0JS0jM2cPKeXcsEUIwstbWcul/iO0vuE?= =?us-ascii?Q?ON1Vy3dDlArtDX0Vu8XYeCHHUnmuHGsu6pfJqC27nfRK+DjN4TcY7nRQs2b1?= =?us-ascii?Q?CbuY6ry6fp3dTIDMSlgkPIaUc3pNNcig23btWwHgGLZVrfC/M2lMMFZCv5dG?= =?us-ascii?Q?32IO+dq/c/Oz4Z5hqRxbrtp6I4F7nQHeR41mATMx/jwpystwb2gKVenRcHDp?= =?us-ascii?Q?rWnqjCtr2YCrBcr4zIVIkB36O8aDs+tGlU8a+d5EDLeF+coDKAkikN9tvg9Y?= =?us-ascii?Q?Z7BIE/Pbrpw5QiagcCdtp07dyaoxARdAMcMn1MhSoUDN8L+?= X-Microsoft-Antispam-Message-Info: SlLozp/oA0FtSgD5VDVv940BrhENqoMCke2V/hERQBEMV3v40sY2j4mdmWTaO+r37p/5xMi5hPWi79Ssi8vUa6zrSvsjshn67OMMaqYXNb7QcWLpHECoqj+F5wrIrc3gao9X5G/vDKMu4JtUSmXZnCra0c3SiTYQMeBpJ0LJQevFk3bbxe9Qc1sV8cLQGeVilSx7379IApmPkjZpnpPk3SrYN8U7xNnxiKm12rRr9dtWlxQzqUBGGDqdNof1zW+9hR+9HUgGmXkdZwOWmNQyYG+7o3n5oVuCey+3reF9i9dNGSX0O6tWV0ZULAeWf4bOvjxfU6WArv+N8IKdHz7Qc9zgFpMPzA6QuqlD7g/rWV4= X-Microsoft-Exchange-Diagnostics: 1; VI1PR05MB3277; 6:uTweS8j7p68uXMRgOEmcOzvi4nnzTWNE8f+QZ1HKZC7H7wfJNakiVMdpaolO7lIqXcuzk0PknIVQtEaPjdg1jV9DD7pLp8GnJk947IJrlI3f3d/6ysHi/+IltYQdhxZmfF/jkRhwflnuahiy+0rMy3OAB7tn3dmjrY8/TfoD5kGOLXSxjwks/GxB28LfhHJPnFFzxMyLB0C5VJ68IZb158ivUeb0Pioay2oiinWF2MEfVc1sMlOJViMea012x4IESTpZQuPhVJ9DNGIuMOZ/LA1AWLPTYlKUGv2egM/IsIpCqLYxtrxn/cpAMKOwrNTQqOcRFx9l04j7buDZVFx2eg4AR66XCGz2gBnTVNZ9seruhd2Y5HX03fPW2uq9R76Yox8Wy4S2mVehVLBFD5ut/cvApcc0nev74CQz5Ygk9REdILofTkmV9FA+QPbXVQ31Qyz8OX+j3cXHBEWHUkM8Nw==; 5:ZBFQ/JJhKR0Preq6lPSMCpuGM3vHp25ocMRS5LyU9ZwvTMi7tTcfBGLkgGTBSrsHM0/1ZGIiSDeIIc4b0aVDuBuA8zNPqKrFwQIFHlse8g+mEZHY0fppA/DJY8MBSnuvqi4KYYuiOsqEVsLu0ruOhF6ttNBsQWU7tnbYsQk1DxM=; 7:CBRvBPXcYhA+WGn7IkR6j/1Xv7gFAuOJWHSlTBLGmScjb+uayyFO0RCAbbp51lSrBuNLsEUIIHzaCv/ogJwZQBbMjzwsNafxpaVLgbul4Zt3vZ6Y/cdLp70VvNv56+HCvTu/uzdEcic1+9brfc+NLpeprImDsn6dRcvl9SuNxzhS+WzEYrUX0WoMSm0TuyKvrjxGdQpOXOMGIZIm8fq54Qzo3cRiRTB+S7h2Dsgy5OE1uc9YPK+QjdWqjXLx+Tn5 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2018 14:14:08.3020 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 65955843-2b70-49c1-447f-08d632a87933 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB3277 Subject: [dpdk-dev] [PATCH v2 0/7] net/mlx5: e-switch VXLAN encap/decap hardware offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2018 14:14:10 -0000 This patchset adds the VXLAN encapsulation/decapsulation hardware offload feature for E-Switch. A typical use case of tunneling infrastructure is port representors in switchdev mode, with VXLAN traffic encapsulation performed on traffic coming *from* a representor and decapsulation on traffic going *to* that representor, in order to transparently assign a given VXLAN to VF traffic. Since these actions are supported at the E-Switch level, the "transfer" attribute must be set on such flow rules. They must also be combined with a port redirection action to make sense. Since only ingress is supported, encapsulation flow rules are normally applied on a physical port and emit traffic to a port representor. The opposite order is used for decapsulation. Like other mlx5 E-Switch flow rule actions, these ones are implemented through Linux's TC flower API. Since the Linux interface for VXLAN encap/decap involves virtual network devices (i.e. ip link add type vxlan [...]), the PMD dynamically spawns them on a needed basis through Netlink calls. These VXLAN implicitly created devices are called VTEPs (Virtual Tunnel End Points). VXLAN interfaces are dynamically created for each local port of outer networks and then used as targets for TC "flower" filters in order to perform encapsulation. For decapsulation the VXLAN devices are created for each unique UDP-port. These VXLAN interfaces are system-wide, the only one device with given UDP port can exist in the system (the attempt of creating another device with the same UDP local port returns EEXIST), so PMD should support the shared (between PMD instances) device database. Rules samples consideraions: $PF - physical device, outer network $VF - representor for VF, outer/inner network $VXLAN - VTEP netdev name $PF_OUTER_IP - $PF IP (v4 or v6) within outer network $REMOTE_IP - remote peer IP (v4 or v6) within outer network $LOCAL_PORT - local UDP port $REMOTE_PORT - remote UDP port VXLAN VTEP creation with iproute2 (PMD does the same via Netlink): - for encapsulation: ip link add $VXLAN type vxlan dstport $LOCAL_PORT external dev $PF ip link set dev $VXLAN up tc qdisc del dev $VXLAN ingress tc qdisc add dev $VXLAN ingress $LOCAL_PORT for egress encapsulated traffic (note, this is not source UDP port in the VXLAN header, it is just UDP port assigned to VTEP, no practical usage) is selected from available UDP ports automatically in range 30000-60000. - for decapsulation: ip link add $VXLAN type vxlan dstport $LOCAL_PORT external ip link set dev $VXLAN up tc qdisc del dev $VXLAN ingress tc qdisc add dev $VXLAN ingress $LOCAL_PORT is UDP port receiving the VXLAN traffic from outer networks. All ingress UDP traffic with given UDP destination port from ALL existing netdevs is routed by kernel to the $VXLAN net device. While applying the rule the kernel checks the IP parameter withing rule, determines the appropriate underlaying PF and tryes to setup the rule hardware offload. VXLAN encapsulation VXLAN encap rules are applied to the VF ingress traffic and have the VTEP as actual redirection destinations instead of outer PF. The encapsulation rule should provide: - redirection action VF->PF - VF port ID - some inner network parameters (MACs) - the tunnel outer source IP (v4/v6), (IS A MUST) - the tunnel outer destination IP (v4/v6), (IS A MUST). - VNI - Virtual Network Identifier (IS A MUST) VXLAN encapsulation rule sample for tc utility: tc filter add dev $VF protocol all parent ffff: flower skip_sw \ action tunnel_key set dst_port $REMOTE_PORT \ src_ip $PF_OUTER_IP dst_ip $REMOTE_IP id $VNI action mirred egress redirect dev $VXLAN VXLAN encapsulation rule sample for testpmd: - Setting up outer properties of VXLAN tunnel: set vxlan ip-version ipv4 vni $VNI udp-src $IGNORED udp-dst $REMOTE_PORT ip-src $PF_OUTER_IP ip-dst $REMOTE_IP eth-src $IGNORED eth-dst $REMOTE_MAC - Creating a flow rule on port ID 4 performing VXLAN encapsulation with the above properties and directing the resulting traffic to port ID 0: flow create 4 ingress transfer pattern eth src is $INNER_MAC / ipv4 / udp dst is 4789 / end actions vxlan_encap / port_id id 0 / end There is no direct way found to provide kernel with all required encapsulatioh header parameters. The encapsulation VTEP is created attached to the outer interface and assumed as default path for egress encapsulated traffic. The outer tunnel IP address are assigned to interface using Netlink, the implicit route is created like this: ip addr add peer dev scope link The peer address option provides implicit route, and scope link attribute reduces the risk of conflicts. At initialization time all local scope link addresses are flushed from the outer network device. The destination MAC address is provided via permenent neigh rule: ip neigh add dev lladdr to nud permanent At initialization time all neigh rules of permanent type are flushed from the outer network device. VXLAN decapsulation VXLAN decap rules are applied to the ingress traffic of VTEP ($VXLAN) device instead of PF. The decapsulation rule should provide: - redirection action PF->VF - VF port ID as redirection destination - $VXLAN device as ingress traffic source - the tunnel outer source IP (v4/v6), (optional) - the tunnel outer destination IP (v4/v6), (IS A MUST) - the tunnel local UDP port (IS A MUST, PMD looks for appropriate VTEP with given local UDP port) - VNI - Virtual Network Identifier (IS A MUST) VXLAN decap rule sample for tc utility: tc filter add dev $VXLAN protocol all parent ffff: flower skip_sw \ enc_src_ip $REMOTE_IP enc_dst_ip $PF_OUTER_IP enc_key_id $VNI \ enc_dst_port $LOCAL_PORT \ action tunnel_key unset action mirred egress redirect dev $VF VXLAN decap rule sample for testpmd: - Creating a flow on port ID 0 performing VXLAN decapsulation and directing the result to port ID 4 with checking inner properties: flow create 1 ingress transfer pattern eth src is 66:77:88:99:aa:bb dst is 00:11:22:33:44:55 / ipv4 src is $REMOTE_IP dst $PF_LOCAL_IP / udp src is 9999 dst is $LOCAL_PORT / vxlan vni is $VNI / end actions vxlan_decap / port_id id 2 / end The VXLAN encap/decap rules constrains (implied by current kernel support) - VXLAN decapsulation provided for PF->VF direction only - VXLAN encapsulation provided for VF->PF direction only - current implementation will support non-shared database of VTEPs (impossible simultaneous usage of the same UDP port by several instances of DPDK apps) Suggested-by: Adrien Mazarguil Signed-off-by: Viacheslav Ovsiienko --- v2: * removed non-VXLAN related parts * multipart Netlink messages support * local IP and peer IP rules management * neigh IP address to MAC address rules management * rules cleanup at outer device initialization * attached devices cleanup at outer device initialization v1: * http://patches.dpdk.org/patch/45800/ * Refactored code of initial experimental proposal v0: * http://patches.dpdk.org/cover/44080/ * Initial proposal by Adrien Mazarguil ** BLURB HERE *** Viacheslav Ovsiienko (7): net/mlx5: e-switch VXLAN configuration and definitions net/mlx5: e-switch VXLAN flow validation routine net/mlx5: e-switch VXLAN flow translation routine net/mlx5: e-switch VXLAN netlink routines update net/mlx5: e-switch VXLAN tunnel devices management net/mlx5: e-switch VXLAN encapsulation rules management net/mlx5: e-switch VXLAN rule cleanup routines drivers/net/mlx5/Makefile | 80 + drivers/net/mlx5/meson.build | 32 + drivers/net/mlx5/mlx5_flow.h | 11 + drivers/net/mlx5/mlx5_flow_tcf.c | 3285 +++++++++++++++++++++++++++++++++++--- 4 files changed, 3165 insertions(+), 243 deletions(-) -- 1.8.3.1