From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0078.outbound.protection.outlook.com [104.47.0.78]) by dpdk.org (Postfix) with ESMTP id 700471B2C1 for ; Thu, 18 Jan 2018 14:38:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=e2btpvvg9nJqbTiNU9eKFVBZ9m1mBCBYxNNlR2nNZI8=; b=J9saFrHC6TMNSDGIFTgAoy0IMZHHdwn0l+JvVtMBi15v1rZ4Rdga0EexSC67HE+FUL8RzA5mBoIGB2OQHJat5Hinf7BpRKsmWj0uTnb2EGq9otoJ5n/90fEY29+0u0eGD5aKqfIczLzdsSvqxy59+8L9gPovsuIfNxg0Ha0l8b4= Received: from mellanox.com (37.142.13.130) by VI1PR0502MB3886.eurprd05.prod.outlook.com (2603:10a6:803:c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.407.7; Thu, 18 Jan 2018 13:38:34 +0000 From: Ophir Munk To: dev@dpdk.org, Pascal Mazon Cc: Thomas Monjalon , Olga Shern , Ophir Munk Date: Thu, 18 Jan 2018 13:38:08 +0000 Message-Id: <1516282691-29698-4-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1516282691-29698-1-git-send-email-ophirmu@mellanox.com> References: <1515692751-25257-1-git-send-email-ophirmu@mellanox.com> <1516282691-29698-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR0802CA0042.eurprd08.prod.outlook.com (2603:10a6:800:a9::28) To VI1PR0502MB3886.eurprd05.prod.outlook.com (2603:10a6:803:c::11) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 98fc49c5-15b3-41b0-c2bd-08d55e78c5e6 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:VI1PR0502MB3886; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3886; 3:yB2/F2ZW69B8QFCwJvQllq2xNE6VoR9rnTvRwEtU/wKB4HrjcIktotjXrF08rn7vnlg4w6Ia1e/ZZy96gzf6xOQy1pSlVEP5HRExqhqkzFiH6pyS14p0UpjmFoDlybuEqCvotogxsTdwMxJs/4DEoiAaGtNvwH4zZXbKBxgUAB9PjvVTPQonACVDjREDwZEsCtGezV+LDIOXY9YxDWLST1wRfFcy0xowQLbNcpviPArMmXeygoBXUveoVe9RfMW/; 25:ycRmNA3X0Dp+ktSRU8eTlHr3pkiVnmqUfXDgyNVPJb7RK3TnucnLw75LhI6XtZw7mYy2ytoVZ7fnmLtj2SYi9zHq9N7IgTPUUAwXi4tkF76doR5DqvYwkDZ/HCcfHuf0TULUUEMd540z86q6lUj0twEmjrshb1vrFJ/gpKX6cGim+c4KrPrGpNB2b+2llkQhnBlLi5x+NoHikorEMB8VhDILnAoC15zv9rsoecZuYnxpd96uyvnAUJSruqAbPMALvFMea1n1SHQynKoPQnkv3N+OsIoMkkj1ubTniGhdOa+DxCp2hNOpRxU4dQuUVHzS/tJBry4PxZcKILi1S6/67g==; 31:OC12S02chwVY2z3yTKosylMoMfME+i4x52PT5u14G2iv6TlK82LUTHY0VDSFsR9mpPKRlDsIraF7PRjXJ9mjQDSE45+6G+eYYTFwUclxomyPOhMwucYsoz4u3YhfERPUQShhtDOdTZrwhG+zHzcrO41ASgvuZskH67nN2ZTJw3M+rXTpwCiVbowrMhzA+0yGOv+nDm4/6VZBrHZhTSlf/WDZ6p+YKi0TEUZardH0qBk= X-MS-TrafficTypeDiagnostic: VI1PR0502MB3886: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3886; 20:9bdi5w3JV06mxapb5ZJ7iM4KYQhuoHgPTPAu24S0tZmSzNJdvnTCxyx5hYhbXNIzPs0QKjgVsmEdfGC4BC5hXbA/g2lX6dhoZUFo7+D5h09wNwPfcOIZRnhXLCLyTmxVqrd33aa48yapcmCK1+rNT+SAO7nFlq+BJJleyg0IW8FlmVWqJALFXI8KWnMpnA2K9akYsZxcAeJvxGfWq1sLRqc7i3QSZM7k5stXXNvneL+DdSrDTCWcf6KvCCK8JBillPD1KRf6TA3w3V+XqVij5mC6Ej+HG9WejLOW1QTKfrAuW9RGXemEm5zGwcayKFcHnKvhstkCDYlska75Vw9D0rkESCUOZfzNz0/ZkyvEK80mJyndY0oPuJK/sJlKjkFENZVNZE6OxNqj2PianN3/X65ACfY3RCiKVYmFQEsWlFrAdXP1KXM4xyX4sdCsoh8y5tpoDoWShBxeTOm7q4x2h23qsoKZf4eY2fCx3Bl8XN/n3+4hlS6G706QW6do5Ehm; 4:WsUShtK25jx/cmCE06HQXrITV5NKVblPUy283Uyynn5vEZxAjd6bIPdUOHV3eIikOf9nHxkKrZkc785GjrVJ2yKVJJ0RoRzbcCfP8sPbQAPF7Er23ADuRMuv76yPIFyKt3q4eqSzF5ZzQ3M5tKs/bP/aU0sUgl5ACdDCu0tQ0+GzKJH89lrBjAP9BDJPGv0DxEDH7dSe0kRLgERS/E2bMYfg4D/1v0pzu0+FJHRxJUNESFTeO+vIF/vakvxJiU48ILlNeDz52TTFAHiMsuSVPg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(3231023)(2400062)(944501161)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041268)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(6072148)(201708071742011); SRVR:VI1PR0502MB3886; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:VI1PR0502MB3886; X-Forefront-PRVS: 05568D1FF7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(346002)(39380400002)(376002)(396003)(366004)(39860400002)(189003)(199004)(48376002)(33026002)(305945005)(6116002)(478600001)(3846002)(106356001)(66066001)(50466002)(7736002)(8676002)(81166006)(21086003)(8936002)(97736004)(16526018)(4720700003)(68736007)(6666003)(51416003)(7696005)(52116002)(81156014)(4326008)(36756003)(50226002)(26005)(69596002)(105586002)(53936002)(5660300001)(25786009)(386003)(316002)(47776003)(76176011)(2906002)(55016002)(2950100002)(54906003)(86362001)(16586007)(107886003)(6916009); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0502MB3886; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0502MB3886; 23:xrEqkE8UYLjYnqe7lXXD9ka69p4bDIpljOiHAtq?= =?us-ascii?Q?pqwXBJwehH5WgYLjMjfIdDoFF5ues4mgAwqn2yyV28ae7Gi+WqTSP3BeBooo?= =?us-ascii?Q?KfSOeGIU1vE9HW9DLLJYMAG5bQsKGdeT1xam127mS8RKmfwARit/cEDKpB3w?= =?us-ascii?Q?yd+/pziUzyRTkpWWJ8Piz0PlmrMgr7WhSbGOnk5geFo2nHGce8hqExQ0WoQP?= =?us-ascii?Q?Ii2gwOdBDCbad1tIekWDmhpvJgs0wSN0gXn/48F870l83io0luQ5ZFkrHi20?= =?us-ascii?Q?b74Oq9VlAIjzcuZyVweqmtOYXeWui9657JYYC99eqzemZqSdnba/EoK7BtR5?= =?us-ascii?Q?4l4nwBy0AR81HJ+mD3FP0xVOQsduis1z7g0vhl3JPRc0+Lk8N+DG/XzEPcvf?= =?us-ascii?Q?wPErSScDQW1CLZKBDi+7BgcKAYcKZ/yBtNkGBpwyNzqPlvJ0FrtyvIlnYtxV?= =?us-ascii?Q?pg77/g0TtctP0rKBvalWMfqysLzeKOFIpWh0nPHxU79NQsKAKbt/0KqJ905i?= =?us-ascii?Q?iY47ALV9rht72d0JiEaTlTG3LpY1rvXYZnMY1/aYXFJVW6HpcLuuvUu956aR?= =?us-ascii?Q?xQrepiBo0B9oI+feDvt+sfUAEMyioBRlNBbtEd1HwqD4stQKVDwg14Cskr/x?= =?us-ascii?Q?ZPQMZaBvJMEBDlIO1MTCLYSW+Mb2HXSICv7PcEpanWZI1gHTpZ6HXXqLRFsP?= =?us-ascii?Q?VWsC53BICEhAD7A8GxFqpzBi1vxcR3kctlNu1cCvvUeb0UJt71pPehMXLxin?= =?us-ascii?Q?wmLtwdte+06RZyHFGEuPDETItx/iZBCeqOmbZh5SCrT4Nz3mqx5UQvU+vs9G?= =?us-ascii?Q?mEVrZw5HsCo16gWaUtYyodh36MVfIG+DWr/cmSBITpQ1+5KPp+0AlDuz8/hF?= =?us-ascii?Q?3VHzb13s8I/l6JpRXrHseHzx7+ODRcMCga18oOXsKYVAdMm96luXnfO+hZZg?= =?us-ascii?Q?lRsNHbP5/pa+sEP3bn4kXBQyvtAG/YVp6KE5UjZlvBSdUnXzS/op0cLfJ+Ga?= =?us-ascii?Q?obdQS7hLmuzSVYM+3oBJ5gffoKivSx0j26MYc/rgaX4Pw+zmJpdsLeKDbEId?= =?us-ascii?Q?747VzYaoXsTiIRU+6T0AizHDXK6ZRWcXArQ3YWkGqCwh/sNDyhWmAmmObi6f?= =?us-ascii?Q?5ZgHmcktMKCeFQqMhxbxNCJph5MqNIfS98uvKJjQv4stcMIi72X1ObfFbhQa?= =?us-ascii?Q?yt1D+u0L5ZTCzDrNfxyB/DhDDmRAZ1nznAk2m?= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3886; 6:7XZF8vXmjn/9bbVVn8dbzWE4XpUoMV6WtLggq5VbUTRG2+N47arhd3zlO1AQwc6DhOeSVIsfPXsMt0Kk5f/fnzKFT/XYGPFXmxRdqiRmeivZckgeq+uhSAKipt5CNh7g+1SURxvscWCwgml2z5rB5pLGIstg4d+/1V+eZbOEFZGISDqOICSk7RCq7k6nQMnNmQvic7/TecZz5Fzj//VFIRWFf/jA7h+XcD6XfEtf3P1u3ouM85mC4X0UARBe8MYdvABR3An0CED/R21a+VQqh7h1pcIuxCul7llZRl7SjKnq6/tIPPYFfRZzhoOUvrto/vCPCPTd44rTZZruR68J/ADlVnUbT2ttSfAaezs802M=; 5:0pB0M2d0m3E8+cnphfYwZ3LGo11UZ8qOfLNPwkbuA525KDMRj1GUNIF9gdIoht+z2AH+ESUjiVxWIVSon7YLF2Q6pLn9niNUbobKirsrY2b3MQCtwFTwUeTY9wUXpv8ZoIbbL8ioYKwRfX+oZ2BBzQRFjtdHig8dseVrZUxUHao=; 24:6LnNO+WHDvPvropdazgsMxgUPAn1KZ2Wvvhr3TSqlOjIAUSU5dGO3kauzEU8i0ZVROhafoZ3wxgwTO6JM6ukc9E7tscJv/SNTUXupOBncug=; 7:uS0BgJ0cXhzjCGCJqVML4n/bvtWkdwJGkSx0+n2VzoB/Whh7lJxezPqGVxejeqw+SfnZEg/2YklQcaWflmuFQJlOuly6bGyLMj4IJh/WiHw+pxaH5n3ML/HMqc94Jy+yZlUwvY8oPhBNFvllgc92lNPDUz+9g8UTQgSfwPqtxhBOsWLsfY4g022+O8SL33EMI5Up6TkYupu9nOXL6YPR5rD9kWpYBzu7nV1r3K17QLjZi4Ma0lSFAZ/cFn3Co2x/ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2018 13:38:34.6176 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 98fc49c5-15b3-41b0-c2bd-08d55e78c5e6 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0502MB3886 Subject: [dpdk-dev] [PATCH v5 3/6] net/tap: add eBPF program file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 13:38:40 -0000 File tap_bpf_program.c was added with two ELF sections corresponding to two BPF programs and one BPF map. It is the C source file used to generate the eBPF bytes code (see "net/tap: add eBPF bytes code"). Section cls_q - BPF classifier to classify packets to their corresponding queue after an RSS hash was calculated on the packet and saved in skb->cb[1] Section l3_l4 - BPF action to calculate RSS hash on packet layers 3 and 4 This file is not part of DPDK tree compilation. Signed-off-by: Ophir Munk --- drivers/net/tap/tap_bpf_program.c | 221 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 drivers/net/tap/tap_bpf_program.c diff --git a/drivers/net/tap/tap_bpf_program.c b/drivers/net/tap/tap_bpf_program.c new file mode 100644 index 0000000..848c50b --- /dev/null +++ b/drivers/net/tap/tap_bpf_program.c @@ -0,0 +1,221 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 + * Copyright 2017 Mellanox Technologies, Ltd. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "tap_rss.h" + +/** Create IPv4 address */ +#define IPv4(a, b, c, d) ((__u32)(((a) & 0xff) << 24) | \ + (((b) & 0xff) << 16) | \ + (((c) & 0xff) << 8) | \ + ((d) & 0xff)) + +#define PORT(a, b) ((__u16)(((a) & 0xff) << 8) | \ + ((b) & 0xff)) + +/* + * The queue number is offset by 1, to distinguish packets that have + * gone through this rule (skb->cb[1] != 0) from others. + */ +#define QUEUE_OFFSET 1 +#define PIN_GLOBAL_NS 2 + +#define KEY_IDX 0 +#define BPF_MAP_ID_KEY 1 + +struct vlan_hdr { + __be16 proto; + __be16 tci; +}; + +struct bpf_elf_map __attribute__((section("maps"), used)) +map_keys = { + .type = BPF_MAP_TYPE_HASH, + .id = BPF_MAP_ID_KEY, + .size_key = sizeof(__u32), + .size_value = sizeof(struct rss_key), + .max_elem = 256, + .pinning = PIN_GLOBAL_NS, +}; + +__section("cls_q") int +match_q(struct __sk_buff *skb) +{ + __u32 queue = skb->cb[1]; + volatile __u32 q = 0xdeadbeef; + __u32 match_queue = QUEUE_OFFSET + q; + + /* printt("match_q$i() queue = %d\n", queue); */ + + if (queue != match_queue) + return TC_ACT_OK; + return TC_ACT_UNSPEC; +} + + +struct ipv4_l3_l4_tuple { + __u32 src_addr; + __u32 dst_addr; + __u16 dport; + __u16 sport; +} __attribute__((packed)); + +struct ipv6_l3_l4_tuple { + __u8 src_addr[16]; + __u8 dst_addr[16]; + __u16 dport; + __u16 sport; +} __attribute__((packed)); + +static const __u8 def_rss_key[] = { + 0xd1, 0x81, 0xc6, 0x2c, + 0xf7, 0xf4, 0xdb, 0x5b, + 0x19, 0x83, 0xa2, 0xfc, + 0x94, 0x3e, 0x1a, 0xdb, + 0xd9, 0x38, 0x9e, 0x6b, + 0xd1, 0x03, 0x9c, 0x2c, + 0xa7, 0x44, 0x99, 0xad, + 0x59, 0x3d, 0x56, 0xd9, + 0xf3, 0x25, 0x3c, 0x06, + 0x2a, 0xdc, 0x1f, 0xfc, +}; + +static __u32 __attribute__((always_inline)) +rte_softrss_be(const __u32 *input_tuple, const uint8_t *rss_key, + __u8 input_len) +{ + __u32 i, j, hash = 0; +#pragma unroll + for (j = 0; j < input_len; j++) { +#pragma unroll + for (i = 0; i < 32; i++) { + if (input_tuple[j] & (1 << (31 - i))) { + hash ^= ((const __u32 *)def_rss_key)[j] << i | + (__u32)((uint64_t) + (((const __u32 *)def_rss_key)[j + 1]) + >> (32 - i)); + } + } + } + return hash; +} + +static int __attribute__((always_inline)) +rss_l3_l4(struct __sk_buff *skb) +{ + void *data_end = (void *)(long)skb->data_end; + void *data = (void *)(long)skb->data; + __u16 proto = (__u16)skb->protocol; + __u32 key_idx = 0xdeadbeef; + __u32 hash; + struct rss_key *rsskey; + __u64 off = ETH_HLEN; + int j; + __u8 *key = 0; + __u32 len; + __u32 queue = 0; + + rsskey = map_lookup_elem(&map_keys, &key_idx); + if (!rsskey) { + printt("hash(): rss key is not configured\n"); + return TC_ACT_OK; + } + key = (__u8 *)rsskey->key; + + /* Get correct proto for 802.1ad */ + if (skb->vlan_present && skb->vlan_proto == htons(ETH_P_8021AD)) { + if (data + ETH_ALEN * 2 + sizeof(struct vlan_hdr) + + sizeof(proto) > data_end) + return TC_ACT_OK; + proto = *(__u16 *)(data + ETH_ALEN * 2 + + sizeof(struct vlan_hdr)); + off += sizeof(struct vlan_hdr); + } + + if (proto == htons(ETH_P_IP)) { + if (data + off + sizeof(struct iphdr) + sizeof(__u32) + > data_end) + return TC_ACT_OK; + + __u8 *src_dst_addr = data + off + offsetof(struct iphdr, saddr); + __u8 *src_dst_port = data + off + sizeof(struct iphdr); + struct ipv4_l3_l4_tuple v4_tuple = { + .src_addr = IPv4(*(src_dst_addr + 0), + *(src_dst_addr + 1), + *(src_dst_addr + 2), + *(src_dst_addr + 3)), + .dst_addr = IPv4(*(src_dst_addr + 4), + *(src_dst_addr + 5), + *(src_dst_addr + 6), + *(src_dst_addr + 7)), + .sport = PORT(*(src_dst_port + 0), + *(src_dst_port + 1)), + .dport = PORT(*(src_dst_port + 2), + *(src_dst_port + 3)), + }; + __u8 input_len = sizeof(v4_tuple) / sizeof(__u32); + if (rsskey->hash_fields & (1 << HASH_FIELD_IPV4_L3)) + input_len--; + hash = rte_softrss_be((__u32 *)&v4_tuple, key, 3); + } else if (proto == htons(ETH_P_IPV6)) { + if (data + off + sizeof(struct ipv6hdr) + + sizeof(__u32) > data_end) + return TC_ACT_OK; + __u8 *src_dst_addr = data + off + + offsetof(struct ipv6hdr, saddr); + __u8 *src_dst_port = data + off + + sizeof(struct ipv6hdr); + struct ipv6_l3_l4_tuple v6_tuple; + for (j = 0; j < 4; j++) + *((uint32_t *)&v6_tuple.src_addr + j) = + __builtin_bswap32(*((uint32_t *) + src_dst_addr + j)); + for (j = 0; j < 4; j++) + *((uint32_t *)&v6_tuple.dst_addr + j) = + __builtin_bswap32(*((uint32_t *) + src_dst_addr + 4 + j)); + v6_tuple.sport = PORT(*(src_dst_port + 0), + *(src_dst_port + 1)); + v6_tuple.dport = PORT(*(src_dst_port + 2), + *(src_dst_port + 3)); + + __u8 input_len = sizeof(v6_tuple) / sizeof(__u32); + if (rsskey->hash_fields & (1 << HASH_FIELD_IPV6_L3)) + input_len--; + hash = rte_softrss_be((__u32 *)&v6_tuple, key, 9); + } else { + return TC_ACT_PIPE; + } + + queue = rsskey->queues[(hash % rsskey->nb_queues) & + (TAP_MAX_QUEUES - 1)]; + skb->cb[1] = QUEUE_OFFSET + queue; + /* printt(">>>>> rss_l3_l4 hash=0x%x queue=%u\n", hash, queue); */ + + return TC_ACT_RECLASSIFY; +} + +#define RSS(L) \ + __section(#L) int \ + L ## _hash(struct __sk_buff *skb) \ + { \ + return rss_ ## L (skb); \ + } + +RSS(l3_l4) + +BPF_LICENSE("Dual BSD/GPL"); -- 2.7.4