From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 75B15A00C5; Sun, 5 Jul 2020 16:51:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 53E0C1DA55; Sun, 5 Jul 2020 16:51:08 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id E32AA1DA54 for ; Sun, 5 Jul 2020 16:51:06 +0200 (CEST) Received: from mx1-us1.ppe-hosted.com (unknown [10.110.50.150]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 8A4962005B; Sun, 5 Jul 2020 14:51:06 +0000 (UTC) Received: from us4-mdac16-13.at1.mdlocal (unknown [10.110.49.195]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 8760F800A3; Sun, 5 Jul 2020 14:51:06 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.110.49.107]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id EFAE9100061; Sun, 5 Jul 2020 14:51:05 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 897F7280057; Sun, 5 Jul 2020 14:51:05 +0000 (UTC) Received: from [192.168.38.17] (10.17.10.39) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 5 Jul 2020 15:50:59 +0100 To: Gregory Etelson , CC: , , Eli Britstein , Ori Kam References: <20200625160348.26220-1-getelson@mellanox.com> <20200625160348.26220-3-getelson@mellanox.com> From: Andrew Rybchenko Autocrypt: addr=arybchenko@solarflare.com; keydata= mQINBF2681gBEACbdTxu8eLL3UX2oAelsnK9GkeaJeUYSOHPJQpV7RL/iaIskqTwBRnhjXt7 j9UEwGA+omnOmqQMpeQTb/F9Ma2dYE+Hw4/t/1KVjxr3ehFaASvwR4fWJfO4e2l/Rk4rG6Yi 5r6CWU2y8su2654Fr8KFc+cMGOAgKoZTZHZsRy5lHpMlemeF+VZkv8L5sYJWPnsypgqlCG3h v6lbtfZs+QqYbFH6bqoZwBAl5irmxywGR7ZJr1GLUZZ1lfdazSY8r6Vz0/Ip/KVxGu2uxo81 QCsAj0ZsQtwji9Sds/prTiPrIjx8Fc/tfbnAuVuPcnPbczwCJACzQr4q26XATL3kVuZhSBWh 4XfO/EAUuEq5AemUG5DDTM87g7Lp4eT9gMZB6P+rJwWPNWTiV3L7Cn+fO+l9mTPnOqdzBgDe OaulKiNSft1o0DY4bGzOmM2ad2cZt0jfnbMPMTE9zsr6+RFa+M8Ct20o6U1MUE4vP6veErMK of4kZ8PdoMM+Sq1hxMPNtlcVBSP9xMmdSZPlfDYI5VWosOceEcz7XZdjBJKdwKuz70V7eac4 ITSxgNFCTbeJ03zL2MR5s0IvD9ghISAwZ6ieCjU5UATn5+63qpD0nVNLsAdb/UpfvQcKAmvj 0fKlxu/PMVkjBa7/4cfNogYOhWDKUO+1pMaFwvb6/XTo6uMpfQARAQABtCxBbmRyZXcgUnli Y2hlbmtvIDxhcnliY2hlbmtvQHNvbGFyZmxhcmUuY29tPokCVAQTAQoAPhYhBP6NPgcKRj/Y X0yXQahue0sAy4m+BQJduvNYAhsDBQkB4TOABQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKhue0sAy4m+t3gP/j1MNc63CEozZo1IZ2UpVPAVWTYbLdPjIRdFqhlwvZYIgGIgIBk3ezKL K0/oc4ZeIwL6wQ5+V24ahuXvvcxLlKxfbJ6lo2iQGC7GLGhsDG9Y2k6sW13/sTJB/XuR2yov k5FtIgJ+aHa1PDZnepnGGOt9ka9n/Jzrc9WKYapOIIyLRe9U26ikoVgyqsD37PVeq5tLWHHA NGTUKupe9G6DFWidxx0KzyMoWDTbW2AWYcEmV2eQsgRT094AZwLFN5ErfefYzsGdO8TAUU9X YTiQN2MvP1pBxY/r0/5UfwV4UKBcR0S3ZvzyvrPoYER2Kxdf/qurx0Mn7StiCQ/JlNZb/GWQ TQ7huduuZHNQKWm7ufbqvKSfbPYvfl3akj7Wl8/zXhYdLqb5mmK45HXrgYGEqPN53OnK2Ngx IgYKEWr05KNv09097jLT5ONgYvszflqlLIzC4dV245g7ucuf9fYmsvmM1p/gFnOJBJL18YE5 P1fuGYNfLP+qp4WMiDqXlzaJfB4JcinyU49BXUj3Utd6f6sNBsO8YWcLbKBV9WmA324S3+wj f4NPRp3A5E+6OmTVMLWire2ZvnYp3YvifUj1r8lhoZ2B2vKuWwiTlHOKYBEjnOQJQnqYZEF0 JQQ1xzVDBQKE01BPlA3vy6BGWe6I4psBVqMOB9lAev/H+xa4u6Z3uQINBF269JsBEAC2KB3W 8JES/fh74avN7LOSdK4QA7gFIUQ4egVL81KnxquLzzilABuOhmZf3Rq6rMHSM8xmUAWa7Dkt YtzXStjEBI/uF0mAR3mMz1RcL2Wp+WD/15HjVpA7hPjXSEsWY0K2ymPerK4yrLcfFTHdMonY JfuACCC9NtOZxrWHOJoUS+RT7AWk80q/6D2iwQ47/2dBTznVG+gSeHSes9l91TB09w6f9JX/ sT+Ud0NQfm7HJ7t2pmGI9O6Po/NLZsDogmnIpJp/WwYOZN9JK7u2FyX2UyRzR8jK42aJkRsh DXs16Cc2/eYGakjrdO3x9a+RoxN7EuFtYhGR1PzMXdUiB5i+FyddYXkYUyO43QE/3VPA5l1v TUOagzZq6aONsdNonGJkV3TIG3JmUNtM+D/+r6QKzmgoJ8w576JxEZI09I/ZFN+g7BnUmlMx 6Z3IUOXVX/SWfGFga0YajwajHz03IBhChEbYbbqndVhmshu2GFURxrfUPYWdDXEqkh+08a5U Didia9jm2Opv4oE1e1TXAePyYJl/Zyps4Cv00GObAxibvMBQCUZQ+IBnNldRBOwXXRQV2xpx P+9iO1VYA/QXn0KqRK+SH1JGRXbJYi42YFaW1gE0EU0fiR2Wb9pK+doNEjjOhlzUGuvOEAUS +4m0m3dlfEvpCV9GMr7ERRpZzh9QkQARAQABiQI8BBgBCgAmFiEE/o0+BwpGP9hfTJdBqG57 SwDLib4FAl269JsCGwwFCQlmAYAACgkQqG57SwDLib7x6g//e+eCtNnJz7qFGbjWRJYNLCe5 gQwkhdyEGk4omr3VmjGj3z9kNFy/muh4pmHUngSAnnpwZggx14N4hhKf9y8G4Dwvsqa6b1zB Jq/c4t/SBDtGW4M/E331N04PaQZpcrbTfp1KqHNknk2N7yOk4CcoLVuIZmA5tPguASV8aAfz ZwhWAwn6vUEw9552eXEAnGFGDTCbyryNwzB5jtVQOEEDjTxcCkpcXMB45Tb1QUslRTu/sBAe HhPCQSUcJHR+KOq+P6yKICGAr291PZd6Qc7C3UyE+A3pY/UfdEVWj0STBWx1qvYLaHLrI4O9 KXDgh7luLjZZafcueCaPYmNo4V2lmNb3+7S4TvqhoZS+wN+9ldRQ4gH3wmRZybN6Y/ZCqxol RaZpE3AqdWsGvIgAkD0FpmtZNii9s2pnrhw0K6S4t4tYgXGTossxNSJUltfFQZdXM1xkZhtv dBZuUEectbZWuviGvQXahOMuH2pM64mx2hpdZzPcI2beeJNHkAsGT2KcaMETgvtHUBFRlLVB YxsUYz3UZmi2JSua4tbcGd6iWVN90eb8CxszYtivfpz6o2nPSjNwg0NaVGSHXjAK0tdByZ9t SkwjC3tEPljVycRSDpbauogOiAkvjENfaPd/H26V5hY822kaclaKDAW6ZG9UKiMijcAgb9u5 CJoOyqE8aGS5Ag0EXbr1RwEQAMXZHbafqmZiu6Kudp+Filgdkj2/XJva5Elv3fLfpXvhVt0Y if5Rzds3RpffoLQZk9nPwK8TbZFqNXPu7HSgg9AY7UdCM94WRFTkUCGKzbgiqGdXZ7Vyc8cy teGW+BcdfQycDvjfy50T3fO4kJNVp2LDNdknPaZVe8HJ80Od63+9ksB6Ni+EijMkh6Uk3ulB CSLnT4iFV57KgU2IsxOQVLnm+0bcsWMcCnGfphkY0yKP+aJ6MfmZkEeaDa7kf24N14ktg50m vOGDitcxA/+XXQXOsOIDJx1VeidxYsQ2FfsKu1G8+G6ejuaLf4rV5MI/+B/tfLbbOdikM5PF pxZVgTir9q13qHumMxdme7w5c7hybW412yWAe9TsrlXktFmFjRSFzAAxQhQSQxArS6db4oBk yeYJ59mW52i4occkimPWSm/raSgdSM+0P6zdWUlxxj+r1qiLgCYvruzLNtp5Nts5tR/HRQjE /ohQYaWDSVJEsc/4eGmgwzHzmvHtXeKkasn01381A1Lv3xwtpnfwERMAhxBZ8EGKEkc5gNdk vIPhknnGgPXqKmE1aWu8LcHiY+RHAF8gYPCDMuwyzBYnbiosKcicuIUp0Fj8XIaPao6F+WTi In4UOrqrYhsaCUvhVjsTBbNphGih9xbFJ8E+lkTLL8P3umtTcMPnpsB4xqcDABEBAAGJBHIE GAEKACYWIQT+jT4HCkY/2F9Ml0GobntLAMuJvgUCXbr1RwIbAgUJCWYBgAJACRCobntLAMuJ vsF0IAQZAQoAHRYhBNTYjdjWgdaEN5MrAN+9UR5r/4d3BQJduvVHAAoJEN+9UR5r/4d3EiQP /3lyby6v49HTU94Q2Fn2Xat6uifR7kWE5SO/1pUwYzx6v+z5K2jqPgqUYmuNoejcGl0CTNhg LbsxzUmAuf1OTAdE+ZYvOAjjKQhY4haxHc4enby/ltnHfWJYWJZ9UN5SsIQLvITvYu6rqthO CYjpXJhwkj3ODmC9H1TrvjrBGc6i7CTnR8RCjMEwCs2LI2frHa4R6imViEr9ScMfUnzdABMQ B0T5MOg8NX92/FRjTldU2KovG0ML9mSveSvVHAoEBLy4UIs5nEDdNiO1opJgKb5CXvWQugub 7AR52phNdKVdEB0S4tigJT4NalyTaPiUhFEm+CzZpMQDJ5E+/OowaPRfN4HeJX+c8sB+vUAZ mkAaG75N+IEk5JKFK9Z+bBYgPgaBDFZYdWDB/TMH0ANt+KI5uYg0i12TB4M8pwKG1DEPUmWc F2YpvB3jnbwzsOpSFiJOOlSs6nOB0Sb5GRtPOO3h6XGj+6mzQd6tcL63c9TrrUkjq7LDkxCz SJ2hTYRC8WNX8Uw9skWo5728JNrXdazEYCenUWmYiKLNKLslXCFodUCRDh/sUiyqRwS7PHEA LYC/UIWLMomI0Yvju3KA5v3RQVXhL+Gx2CzSj3GDz9xxGhJB2LfRfjzPbTR/Z27UpjCkd8z0 Ro3Ypmi1FLQwnRgoOKDbetTAIhugEShaLTITzJAP/iRDJCQsrZah5tE8oIl81qKEmBJEGcdt HYikbpQe7ydcXhqTj7+IECa3O7azI5OhCxUH2jNyonJ/phUslHH2G1TTBZK8y4Hrx5RpuRNS esn3P9uKu9DHqBAL7DMsCPwb2p1VNnapD72DBmRhzS/e6zS2R4+r9yNv03Hv7VCxKkmtE63H qpS//qpjfrtsIcHAjnKDaDtL1LYCtHoweI+DOpKKULSAYp/JE6F8LNibPQ0/P3S5ZIJNC4QZ uESjFOalJwFIqGQdkQB7ltRNJENLrHc+2jKGOuyFHm/Sbvp5EMGdaeQ0+u8CY0P+y6oXenwx 7WrJz/GvbNoFhJoJ6RzxCMQrFgxrssVZ7w5HcUj94lbnJ6osdYE/WpSd50B6jet6LKh5revg u9XI9CoqsPQ1V4wKYYdllPuogCye7KNYNKuiiuSNpaF4gHq1ZWGArwZtWHjgc2v3LegOpRQF SwOskMKmWsUyHIRMG1p8RpkBQTqY2rGSeUqPSvaqjT0nq+SUEM6qxEXD/2Wqri/X6bamuPDb S0PkBvFD2+0zr5Bc2YkMGPBYPNGZiTp3UjmZlLfn3TiBKIC92jherY563CULjSsiBEJCOSvv 4VPLn5aAcfbCXJnE3IGCp/hPl50iQqu7BPOYBbWXeb9ptDjGCAThNxSz0WAXkmcjAFE8gdE6 Znk9 Message-ID: Date: Sun, 5 Jul 2020 17:50:49 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20200625160348.26220-3-getelson@mellanox.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.17.10.39] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25522.003 X-TM-AS-Result: No-15.507600-8.000000-10 X-TMASE-MatchedRID: yebcs53SkkC1TiWqZWCojwCNFKULxGCZPXu1L28jSnEgVZAf8m502Mfl Ru51nE3BLMb8GeIUasIWpG2Xa/CEWz4Pcn5OGAtGIwk7p1qp3JYfgSFHj7MmkH8ozV5Wwb50IGg agoxz9mkaP9WJhp9x6nBV0D37nd4oWNQe1bLu8dnB5xi0VO7ouKdjSOpZb2vXzdRnve/kf9RLUw cMs5f57/XBEDWZLYk1p6hWiMVx/WRcQDmqyguu2Pw4wQxTs7Jm8vvksslXuLcOOOIzzESoE49/b 50cC04oCH6eQxSXw5xyuZFH776M/Bn2ID0UQyAGLXqeYLb/NKr8SjXSxldPU+ZYcdJgScjxIg44 gdXGnvmkP9glHMOS9cSIUbXCYKjGv8IgVRXcmad2GcWKGZufBUYj0zDHPzJpNukyDIB+Hbq7+r6 27MDAzArlDmsKPi+d4ofM/QG5QINeb+a1MNmpRaKg5CrKMLXj+KgiyLtJrSChIJbJvgouD04afT HuzVFfCX0UjEERAwzm9S/0eIQUb4H/9PEJqVb6nFVnNmvv47tLXPA26IG0hN9RlPzeVuQQdZTrO VEffU1ASwhhIShNVIuN1wHRV+njNc2I6aPU2RwdxBAG5/hkW8K2EIRFRDeca0TOsL14A2lbHAgg izNNyCJDH5t2Q95rSPzvv+w86cjxkVx/+7DFCxwu4QM/6Cpyab+ZPXqZNQI1W2FJXnJz9k/ZsGZ 5IDi7CPSQluJKFQVGlyZtRTr6W0DZEbeO64vOGUlF/M3Dxp8Cn5QffvZFlTpsLzzGtYGcMlIKaE HhPmrLEpVi2vdpV+RzbwIflYZ3OXyRIvc0XEOeAiCmPx4NwFkMvWAuahr8+gD2vYtOFhgqtq5d3 cxkNclWyKy6Vx8ye8ctFTTca/caSmi8TobzZWuuQ0k9nJ6kxfzs/b+e/KU= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--15.507600-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25522.003 X-MDID: 1593960666-cRLJtnU0tIvL Subject: Re: [dpdk-dev] [PATCH 2/2] ethdev: tunnel offload model X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Gregory, I'm sorry for the review with toooo many questions without any suggestions on how to answer it. Please, see below. On 6/25/20 7:03 PM, Gregory Etelson wrote: > From: Eli Britstein > > Hardware vendors implement tunneled traffic offload techniques > differently. Although RTE flow API provides tools capable to offload > all sorts of network stacks, software application must reference this > hardware differences in flow rules compilation. As the result tunneled > traffic flow rules that utilize hardware capabilities can be different > for the same traffic. > > Tunnel port offload proposed in [1] provides software application with > unified rules model for tunneled traffic regardless underlying > hardware. > - The model introduces a concept of a virtual tunnel port (VTP). > - The model uses VTP to offload ingress tunneled network traffic  > with RTE flow rules. > - The model is implemented as set of helper functions. Each PMD > implements VTP offload according to underlying hardware offload > capabilities. Applications must query PMD for VTP flow > items / actions before using in creation of a VTP flow rule. > > The model components: > - Virtual Tunnel Port (VTP) is a stateless software object that > describes tunneled network traffic. VTP object usually contains > descriptions of outer headers, tunnel headers and inner headers. > - Tunnel Steering flow Rule (TSR) detects tunneled packets and > delegates them to tunnel processing infrastructure, implemented > in PMD for optimal hardware utilization, for further processing. > - Tunnel Matching flow Rule (TMR) verifies packet configuration and > runs offload actions in case of a match. > > Application actions: > 1 Initialize VTP object according to tunnel > network parameters. > 2 Create TSR flow rule: > 2.1 Query PMD for VTP actions: application can query for VTP actions > more than once > int > rte_flow_tunnel_decap_set(uint16_t port_id, > struct rte_flow_tunnel *tunnel, > struct rte_flow_action **pmd_actions, > uint32_t *num_of_pmd_actions, > struct rte_flow_error *error); > > 2.2 Integrate PMD actions into TSR actions list. > 2.3 Create TSR flow rule: > flow create group 0 > match {tunnel items} / end > actions {PMD actions} / {App actions} / end > > 3 Create TMR flow rule: > 3.1 Query PMD for VTP items: application can query for VTP items > more than once > int > rte_flow_tunnel_match(uint16_t port_id, > struct rte_flow_tunnel *tunnel, > struct rte_flow_item **pmd_items, > uint32_t *num_of_pmd_items, > struct rte_flow_error *error); > > 3.2 Integrate PMD items into TMR items list: > 3.3 Create TMR flow rule > flow create group 0 > match {PMD items} / {APP items} / end > actions {offload actions} / end > > The model provides helper function call to restore packets that miss > tunnel TMR rules to its original state: > int > rte_flow_get_restore_info(uint16_t port_id, > struct rte_mbuf *mbuf, > struct rte_flow_restore_info *info, > struct rte_flow_error *error); > > rte_tunnel object filled by the call inside > rte_flow_restore_info *info parameter can be used by the application > to create new TMR rule for that tunnel. > > The model requirements: > Software application must initialize > rte_tunnel object with tunnel parameters before calling > rte_flow_tunnel_decap_set() & rte_flow_tunnel_match(). > > PMD actions array obtained in rte_flow_tunnel_decap_set() must be > released by application with rte_flow_action_release() call. > Application can release the actionsfter TSR rule was created. > > PMD items array obtained with rte_flow_tunnel_match() must be released > by application with rte_flow_item_release() call. Application can > release the items after rule was created. However, if the application > needs to create additional TMR rule for the same tunnel it will need > to obtain PMD items again. > > Application cannot destroy rte_tunnel object before it releases all > PMD actions & PMD items referencing that tunnel. > > [1] https://mails.dpdk.org/archives/dev/2020-June/169656.html > > Signed-off-by: Eli Britstein > Acked-by: Ori Kam > --- > doc/guides/prog_guide/rte_flow.rst | 105 ++++++++++++ > lib/librte_ethdev/rte_ethdev_version.map | 5 + > lib/librte_ethdev/rte_flow.c | 112 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 196 +++++++++++++++++++++++ > lib/librte_ethdev/rte_flow_driver.h | 32 ++++ > 5 files changed, 450 insertions(+) > > diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst > index d5dd18ce99..cfd98c2e7d 100644 > --- a/doc/guides/prog_guide/rte_flow.rst > +++ b/doc/guides/prog_guide/rte_flow.rst > @@ -3010,6 +3010,111 @@ operations include: > - Duplication of a complete flow rule description. > - Pattern item or action name retrieval. > > +Tunneled traffic offload > +~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Provide software application with unified rules model for tunneled traffic > +regardless underlying hardware. > + > + - The model introduces a concept of a virtual tunnel port (VTP). It looks like it is absolutely abstract concept now, since it is not mentioned/referenced in the header file. It makes it hard to put the description and API together. > + - The model uses VTP to offload ingress tunneled network traffic  > + with RTE flow rules. > + - The model is implemented as set of helper functions. Each PMD > + implements VTP offload according to underlying hardware offload > + capabilities. Applications must query PMD for VTP flow > + items / actions before using in creation of a VTP flow rule. For me it looks like "creation of a VTP flow rule" is not covered yet. Flow rules examples mention it in pattern and actions, but there is no corresponding pattern items and actions. May be I simply misunderstand the idea. > + > +The model components: > + > +- Virtual Tunnel Port (VTP) is a stateless software object that > + describes tunneled network traffic. VTP object usually contains > + descriptions of outer headers, tunnel headers and inner headers. Are inner headers really a part of the tunnel description? > +- Tunnel Steering flow Rule (TSR) detects tunneled packets and > + delegates them to tunnel processing infrastructure, implemented > + in PMD for optimal hardware utilization, for further processing. > +- Tunnel Matching flow Rule (TMR) verifies packet configuration and > + runs offload actions in case of a match. Is it for fully offloaded tunnels with encap/decap or all tunnels (detected, but partially offloaded, e.g. checksumming)? > + > +Application actions: > + > +1 Initialize VTP object according to tunnel network parameters. I.e. fill in 'struct rte_flow_tunnel'. Is it correct? > + > +2 Create TSR flow rule. > + > +2.1 Query PMD for VTP actions. Application can query for VTP actions more than once. > + > + .. code-block:: c > + > + int > + rte_flow_tunnel_decap_set(uint16_t port_id, > + struct rte_flow_tunnel *tunnel, > + struct rte_flow_action **pmd_actions, > + uint32_t *num_of_pmd_actions, > + struct rte_flow_error *error); > + > +2.2 Integrate PMD actions into TSR actions list. > + > +2.3 Create TSR flow rule. > + > + .. code-block:: console > + > + flow create group 0 match {tunnel items} / end actions {PMD actions} / {App actions} / end Are application actions strictly required? If no, it is better to make it clear. Do tunnel items correlate here somehow with tunnel specification in 'struct rte_flow_tunnel'? Is it obtained using rte_flow_tunnel_match()? > + > +3 Create TMR flow rule. > + > +3.1 Query PMD for VTP items. Application can query for VTP items more than once. > + > + .. code-block:: c > + > + int > + rte_flow_tunnel_match(uint16_t port_id, > + struct rte_flow_tunnel *tunnel, > + struct rte_flow_item **pmd_items, > + uint32_t *num_of_pmd_items, > + struct rte_flow_error *error); > + > +3.2 Integrate PMD items into TMR items list. > + > +3.3 Create TMR flow rule. > + > + .. code-block:: console > + > + flow create group 0 match {PMD items} / {APP items} / end actions {offload actions} / end > + > +The model provides helper function call to restore packets that miss > +tunnel TMR rules to its original state: > + > +.. code-block:: c > + > + int > + rte_flow_get_restore_info(uint16_t port_id, > + struct rte_mbuf *mbuf, > + struct rte_flow_restore_info *info, > + struct rte_flow_error *error); > + > +rte_tunnel object filled by the call inside > +``rte_flow_restore_info *info parameter`` can be used by the application > +to create new TMR rule for that tunnel. I think an example, for example, for VXLAN over IPv4 tunnel case with some concrete parameters would be very useful here for understanding. Could it be annotated with a description of the transformations happening with a packet on different stages of the processing (including restore example). > + > +The model requirements: > + > +Software application must initialize > +rte_tunnel object with tunnel parameters before calling > +rte_flow_tunnel_decap_set() & rte_flow_tunnel_match(). > + > +PMD actions array obtained in rte_flow_tunnel_decap_set() must be > +released by application with rte_flow_action_release() call. > +Application can release the actionsfter TSR rule was created. actionsfter ? > + > +PMD items array obtained with rte_flow_tunnel_match() must be released > +by application with rte_flow_item_release() call. Application can > +release the items after rule was created. However, if the application > +needs to create additional TMR rule for the same tunnel it will need > +to obtain PMD items again. > + > +Application cannot destroy rte_tunnel object before it releases all > +PMD actions & PMD items referencing that tunnel. > + > Caveats > ------- > [snip] > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index b0e4199192..1374b6e5a7 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -3324,6 +3324,202 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error *error); > > +/* Tunnel information. */ > +__rte_experimental > +struct rte_flow_ip_tunnel_key { > + rte_be64_t tun_id; /**< Tunnel identification. */ What is it? Why is it big-endian? Why is it in IP tunnel key? I.e. why is it not in a generic structure? > + union { > + struct { > + rte_be32_t src_addr; /**< IPv4 source address. */ > + rte_be32_t dst_addr; /**< IPv4 destination address. */ > + } ipv4; > + struct { > + uint8_t src_addr[16]; /**< IPv6 source address. */ > + uint8_t dst_addr[16]; /**< IPv6 destination address. */ > + } ipv6; > + } u; > + bool is_ipv6; /**< True for valid IPv6 fields. Otherwise IPv4. */ > + rte_be16_t tun_flags; /**< Tunnel flags. */ Which flags? Where are these flags defined? Why is it big-endian? > + uint8_t tos; /**< TOS for IPv4, TC for IPv6. */ > + uint8_t ttl; /**< TTL for IPv4, HL for IPv6. */ If combine, I'd stick to IPv6 terminology since it is a bit better (well-thought, especially current tendencies in (re)naming in software). > + rte_be32_t label; /**< Flow Label for IPv6. */ What about IPv6 tunnels with extension headers? How to extend? > + rte_be16_t tp_src; /**< Tunnel port source. */ > + rte_be16_t tp_dst; /**< Tunnel port destination. */ What about IP-in-IP tunnels? Is it applicable? > +}; > + > + > +/* Tunnel has a type and the key information. */ > +__rte_experimental > +struct rte_flow_tunnel { > + /** > + * Tunnel type, for example RTE_FLOW_ITEM_TYPE_VXLAN, > + * RTE_FLOW_ITEM_TYPE_NVGRE etc. > + */ > + enum rte_flow_item_type type; > + struct rte_flow_ip_tunnel_key tun_info; /**< Tunnel key info. */ How to extended for non-IP tunnels? MPLS? Or tunnels with more protocols? E.g. MPLS-over-UDP? > +}; > + > +/** > + * Indicate that the packet has a tunnel. > + */ > +#define RTE_FLOW_RESTORE_INFO_TUNNEL (1ULL << 0) > + > +/** > + * Indicate that the packet has a non decapsulated tunnel header. > + */ > +#define RTE_FLOW_RESTORE_INFO_ENCAPSULATED (1ULL << 1) > + > +/** > + * Indicate that the packet has a group_id. > + */ > +#define RTE_FLOW_RESTORE_INFO_GROUP_ID (1ULL << 2) > + > +/** > + * Restore information structure to communicate the current packet processing > + * state when some of the processing pipeline is done in hardware and should > + * continue in software. > + */ > +__rte_experimental > +struct rte_flow_restore_info { > + /** > + * Bitwise flags (RTE_FLOW_RESTORE_INFO_*) to indicate validation of > + * other fields in struct rte_flow_restore_info. > + */ > + uint64_t flags; > + uint32_t group_id; /**< Group ID. */ What is the group ID here? > + struct rte_flow_tunnel tunnel; /**< Tunnel information. */ > +}; > + > +/** > + * Allocate an array of actions to be used in rte_flow_create, to implement > + * tunnel-decap-set for the given tunnel. > + * Sample usage: > + * actions vxlan_decap / tunnel-decap-set(tunnel properties) / > + * jump group 0 / end Why is jump to group used in example above? Is it mandatory? > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] tunnel > + * Tunnel properties. > + * @param[out] actions > + * Array of actions to be allocated by the PMD. This array should be > + * concatenated with the actions array provided to rte_flow_create. Please, specify concatenation order explicitly. > + * @param[out] num_of_actions > + * Number of actions allocated. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_tunnel_decap_set(uint16_t port_id, > + struct rte_flow_tunnel *tunnel, > + struct rte_flow_action **actions, > + uint32_t *num_of_actions, Why does approach to specify actions differ here? I.e. array of specified size vs END-terminated array? Must the actions array be END-terminated here? It must be a strong reason to do it and it should be explained. > + struct rte_flow_error *error); > + > +/** > + * Allocate an array of items to be used in rte_flow_create, to implement > + * tunnel-match for the given tunnel. > + * Sample usage: > + * pattern tunnel-match(tunnel properties) / outer-header-matches / > + * inner-header-matches / end > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] tunnel > + * Tunnel properties. > + * @param[out] items > + * Array of items to be allocated by the PMD. This array should be > + * concatenated with the items array provided to rte_flow_create. Concatenation order/rules should be described. Since it is an output which entity does the concatenation. Is it allowed to refine PMD rules in application rule specification? > + * @param[out] num_of_items > + * Number of items allocated. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_tunnel_match(uint16_t port_id, > + struct rte_flow_tunnel *tunnel, > + struct rte_flow_item **items, > + uint32_t *num_of_items, Same as above for actions. > + struct rte_flow_error *error); > + > +/** > + * Populate the current packet processing state, if exists, for the given mbuf. > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] m > + * Mbuf struct. > + * @param[out] info > + * Restore information. Upon success contains the HW state. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_tunnel_get_restore_info(uint16_t port_id, > + struct rte_mbuf *m, > + struct rte_flow_restore_info *info, Is it suggesting to make a copy of the restore info for each mbuf? It sounds very expensive. Could you share your thoughts about it. > + struct rte_flow_error *error); > + > +/** > + * Release the action array as allocated by rte_flow_tunnel_decap_set. > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] actions > + * Array of actions to be released. > + * @param[in] num_of_actions > + * Number of elements in actions array. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_tunnel_action_decap_release(uint16_t port_id, > + struct rte_flow_action *actions, > + uint32_t num_of_actions, Same question as above for actions and items specification approach. > + struct rte_flow_error *error); > + > +/** > + * Release the item array as allocated by rte_flow_tunnel_match. > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] items > + * Array of items to be released. > + * @param[in] num_of_items > + * Number of elements in item array. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_tunnel_item_release(uint16_t port_id, > + struct rte_flow_item *items, > + uint32_t num_of_items, Same question as above for actions and items specification approach. > + struct rte_flow_error *error); > #ifdef __cplusplus > } > #endif [snip] Andrew. (Right now it is hard to fully imagine how to deal with it. And it looks like a shim to vendor-specific API. May be I'm wrong. Hopefully the next version will have PMD implementation example and it will shed a bit more light on it.)