From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 5C2A8DED for ; Sat, 25 Aug 2018 08:11:53 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Aug 2018 23:11:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,285,1531810800"; d="scan'208";a="251797523" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga005.jf.intel.com with ESMTP; 24 Aug 2018 23:11:46 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 24 Aug 2018 23:11:46 -0700 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 24 Aug 2018 23:11:45 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.240]) by shsmsx102.ccr.corp.intel.com ([169.254.2.226]) with mapi id 14.03.0319.002; Sat, 25 Aug 2018 14:11:43 +0800 From: "Zhang, Qi Z" To: "dev@dpdk.org" CC: "Karlsson, Magnus" , "Topel, Bjorn" , "Wu, Jingjing" , "Li, Xiaoyun" , "Yigit, Ferruh" Thread-Topic: [PATCH v3 0/6] PMD driver for AF_XDP Thread-Index: AQHUNW9kNejrbibCBE+27ZaSr3eYaKTP5sUA Date: Sat, 25 Aug 2018 06:11:43 +0000 Message-ID: <039ED4275CED7440929022BC67E706115327A8C1@SHSMSX103.ccr.corp.intel.com> References: <20180816144321.17719-1-qi.z.zhang@intel.com> In-Reply-To: <20180816144321.17719-1-qi.z.zhang@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMGU3MTY4NzUtZDNlZC00YjdkLWJlZDgtZGZiNzU3NDcyYTE1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiZTdjMnZXOEFqRjBUU1B2VTYzXC9YWUdTSmJQYytxem1wdit2Z1hGU21XV2k3d1FyOE9jeUJhN3E2UmJ1RzM0M2wifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 0/6] PMD driver for AF_XDP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Aug 2018 06:11:54 -0000 Sorry, the patch for kernel sample code is not complete. It should be as be= low ~~~~~~~~~~~~~~~~~~~~~~~PATCH START ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index d69c8d78d3fd..44a6318043e7 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -76,6 +76,8 @@ static int opt_poll; static int opt_shared_packet_buffer; static int opt_interval =3D 1; static u32 opt_xdp_bind_flags; +static int opt_bind; +static int opt_unbind; =20 struct xdp_umem_uqueue { u32 cached_prod; @@ -662,6 +664,8 @@ static void usage(const char *prog) " -S, --xdp-skb=3Dn Use XDP skb-mod\n" " -N, --xdp-native=3Dn Enfore XDP native mode\n" " -n, --interval=3Dn Specify statistics update interval (default 1 sec)= .\n" + " -b, --bind Bind only.\n" + " -u, --unbind Unbind only.\n" "\n"; fprintf(stderr, str, prog); exit(EXIT_FAILURE); @@ -674,7 +678,7 @@ static void parse_command_line(int argc, char **argv) opterr =3D 0; =20 for (;;) { - c =3D getopt_long(argc, argv, "rtli:q:psSNn:", long_options, + c =3D getopt_long(argc, argv, "rtli:q:psSNn:bu", long_options, &option_index); if (c =3D=3D -1) break; @@ -711,6 +715,12 @@ static void parse_command_line(int argc, char **argv) case 'n': opt_interval =3D atoi(optarg); break; + case 'b': + opt_bind =3D 1; + break; + case 'u': + opt_unbind =3D 1; + break; default: usage(basename(argv[0])); } @@ -898,6 +908,12 @@ int main(int argc, char **argv) exit(EXIT_FAILURE); } =20 + if (opt_unbind) { + bpf_set_link_xdp_fd(opt_ifindex, -1, opt_xdp_flags); + printf("unbind.\n"); + return 0; + } + snprintf(xdp_filename, sizeof(xdp_filename), "%s_kern.o", argv[0]); =20 if (load_bpf_file(xdp_filename)) { @@ -922,6 +938,11 @@ int main(int argc, char **argv) exit(EXIT_FAILURE); } =20 + if (opt_bind) { + printf("bind.\n"); + return 0; + } + /* Create sockets... */ xsks[num_socks++] =3D xsk_configure(NULL); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~PATCH END~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ BTW, there is bug in kernel bpftool on 4.18, it will cause segment fault wh= en you try to dump bpf map with #./tools/bpf/bpftool/bpftool map -p So, please also apply below patch=20 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~PATCH START ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index 097b1a5e046b..0c661de58976 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -67,6 +67,7 @@ static const char * const map_type_name[] =3D { [BPF_MAP_TYPE_SOCKMAP] =3D "sockmap", [BPF_MAP_TYPE_CPUMAP] =3D "cpumap", [BPF_MAP_TYPE_SOCKHASH] =3D "sockhash", + [BPF_MAP_TYPE_XSKMAP] =3D "xskmap" }; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~PATCH END~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > -----Original Message----- > From: Zhang, Qi Z > Sent: Thursday, August 16, 2018 10:43 PM > To: dev@dpdk.org > Cc: Karlsson, Magnus ; Topel, Bjorn > ; Wu, Jingjing ; Li, Xiaoyu= n > ; Yigit, Ferruh ; Zhang, Qi= Z > > Subject: [PATCH v3 0/6] PMD driver for AF_XDP >=20 > Overview > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > The patch set add a new PMD driver for AF_XDP which is a proposed faster > version of AF_PACKET interface in Linux, see below link for detail AF_XDP > introduction: > https://lwn.net/Articles/750845/ > https://fosdem.org/2018/schedule/event/af_xdp/ >=20 > AF_XDP roadmap > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > - The kernel 4.18 is out and af_xdp is included. > https://kernelnewbies.org/Linux_4.18 > - So far there is no zero copy supported driver be merged, but some are > on the way. >=20 > Change logs > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > v3: > - Re-work base on AF_XDP's interface changes. > - Support multi-queues, each dpdk queue has its own xdp socket. > An xdp socket is always bound to a netdev queue. > We assume all xdp socket from the same ethdev are bound to the > same netdev queue, though a netdev queue still can be bound by > xdp sockets from different ethdev instances. > Below is an example of the mapping. > ------------------------------------------------------ > | dpdk q0 | dpdk q1 | dpdk q0 | dpdk q0 | dpdk q1 | > ------------------------------------------------------ > | xsk A | xsk B | xsk C | xsk D | xsk E |<---| > ------------------------------------------------------ | > | ETHDEV 0 | ETHDEV 1 | ETHDEV 2 | | > DPDK > ------------------------------------------------------------------ > | netdev queue 0 | netdev queue 1 | | > KERNEL > ------------------------------------------------------ | > | NETDEV eth0 | | > ------------------------------------------------------ | > | key xsk | | > | ---------- -------------- | | > | | | | 0 | xsk A | | | > | | | -------------- | | > | | | | 2 | xsk B | | | > | | ebpf | --------------------------------------- > | | | | 3 | xsk C | | > | | redirect ->|-------------- | > | | | | 4 | xsk D | | > | | | -------------- | > | |---------| | 5 | xsk E | | > | -------------- | > |----------------------------------------------------- >=20 > - It is an open question that how to load ebpf to kernel and link to > specific netdev in DPDK, should it be part of PMD, or it should be hand= led > by > an independent tool? In this patchset, it takes the second option, ther= e will > be a "bind" stage before we start AF_XDP PMD, this includes below steps= : > a) load ebpf program to the kernel, (the ebpf program must contain the > logic to redirect packet to a xdp socket base on a redirect map). > b) link ebpf program to specific network interface. > c) expose the xdp socket redirect map id and entries number to user, > so this will be parsed to PMD, and PMD will create xdp socket > for each queue and update the redirect map correctly. > (example: > --vdev,iface=3Deth0,xsk_map_id=3D53,xsk_map_key_base=3D0,xsk_map_key_coun= t > =3D4) >=20 > v2: > - fix lisence header > - clean up bpf dependency, bpf program is embedded, no > "xdpsock_kern.o" > required > - clean up make file, only linux_header is required > - fix all the compile warning. > - fix packet number return in Tx. >=20 > How to try > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > 1. Take the kernel v4.18. > make sure you turn on XDP sockets when compiling > Networking support --> > Networking options --> > [ * ] XDP sockets > 2. in the kernel source code, apply below patch and compile the bpf sampl= e > code. > #make samples/bpf/ > so the sample xdpsock can be used as a bind/unbind tool for af_xdp > PMD, sorry for this ugly, but in future, there could be a dedicated > tool in DPDK, if we agree with the idea that bpf configure in the kern= el > should be separated from PMD. >=20 > ~~~~~~~~~~~~~~~~~~~~~~~PATCH > START~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >=20 > diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c inde= x > d69c8d78d3fd..44a6318043e7 100644 > --- a/samples/bpf/xdpsock_user.c > +++ b/samples/bpf/xdpsock_user.c > @@ -76,6 +76,8 @@ static int opt_poll; > static int opt_shared_packet_buffer; > static int opt_interval =3D 1; > static u32 opt_xdp_bind_flags; > +static int opt_bind; > +static int opt_unbind; >=20 > struct xdp_umem_uqueue { > u32 cached_prod; > @@ -662,6 +664,8 @@ static void usage(const char *prog) > " -S, --xdp-skb=3Dn Use XDP skb-mod\n" > " -N, --xdp-native=3Dn Enfore XDP native mode\n" > " -n, --interval=3Dn Specify statistics update interval (default 1 > sec).\n" > + " -b, --bind Bind only.\n" > + " -u, --unbind Unbind only.\n" > "\n"; > fprintf(stderr, str, prog); > exit(EXIT_FAILURE); > @@ -674,7 +678,7 @@ static void parse_command_line(int argc, char > **argv) > opterr =3D 0; >=20 > for (;;) { > - c =3D getopt_long(argc, argv, "rtli:q:psSNn:", long_options, > + c =3D getopt_long(argc, argv, "rtli:q:psSNn:bu", long_options, > &option_index); > if (c =3D=3D -1) > break; > @@ -711,6 +715,12 @@ static void parse_command_line(int argc, char > **argv) > case 'n': > opt_interval =3D atoi(optarg); > break; > + case 'b': > + opt_bind =3D 1; > + break; > + case 'u': > + opt_unbind =3D 1; > + break; > default: > usage(basename(argv[0])); > } > @@ -898,6 +908,12 @@ int main(int argc, char **argv) > exit(EXIT_FAILURE); > } >=20 > + if (opt_unbind) { > + bpf_set_link_xdp_fd(opt_ifindex, -1, opt_xdp_flags); >=20 > ~~~~~~~~~~~~~~~~~~~~~~~PATCH > END~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >=20 > 3. bind > #./samples/bpf/xdpsock -i eth0 -b >=20 > in this step, an ebpf binary xdpsock_kern.o is be loaded into the kerne= l > and linked to eth0, the ebpf source code is /samples/bpf/xdpsock_kern.c > you can modify it and re-compile for a different test. >=20 > 4. dump xdp socket map information. > #./tools/bpf/bpftool/bpftool map -p, you will see something like below. >=20 > },{ > "id": 56, > "type": "xskmap", > "name": "xsks_map", > "flags": 0, > "bytes_key": 4, > "bytes_value": 4, > "max_entries": 4, > "bytes_memlock": 4096 > } >=20 > in this case 56 is the map id and it has 4 entries >=20 > 5. start testpmd >=20 > ./build/app/testpmd -c 0xc -n 4 --vdev > eth_af_xdp,iface=3Denp59s0f0,xsk_map_id=3D56,xsk_map_key_start=3D2xsk_map= _ > key_count=3D2 -- -i --rxq=3D2 --txq=3D2 >=20 > in this case, we reserved 2 entries (2,3) in the map, and they will b= e > mapped to queue 0 and queue 1. >=20 > 6. unbind after test > ./sample/bpf/xdpsock -i eth0 -u. >=20 > Performance > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Since no zero copy driver is ready yet. > So far only tested with DRV and SKB mode on i40e 25G the result show > identical with kernel sample "xdpsock" >=20 > Qi Zhang (6): > net/af_xdp: new PMD driver > lib/mbuf: enable parse flags when create mempool > lib/mempool: allow page size aligned mempool > net/af_xdp: use mbuf mempool for buffer management > net/af_xdp: enable zero copy > app/testpmd: add mempool flags parameter >=20 > app/test-pmd/parameters.c | 12 + > app/test-pmd/testpmd.c | 15 +- > app/test-pmd/testpmd.h | 1 + > config/common_base | 5 + > config/common_linuxapp | 1 + > drivers/net/Makefile | 1 + > drivers/net/af_xdp/Makefile | 30 + > drivers/net/af_xdp/meson.build | 7 + > drivers/net/af_xdp/rte_eth_af_xdp.c | 1345 > +++++++++++++++++++++++++ > drivers/net/af_xdp/rte_pmd_af_xdp_version.map | 4 + > lib/librte_mbuf/rte_mbuf.c | 15 +- > lib/librte_mbuf/rte_mbuf.h | 8 +- > lib/librte_mempool/rte_mempool.c | 3 + > lib/librte_mempool/rte_mempool.h | 1 + > mk/rte.app.mk | 1 + > 15 files changed, 1439 insertions(+), 10 deletions(-) create mode 10064= 4 > drivers/net/af_xdp/Makefile create mode 100644 > drivers/net/af_xdp/meson.build create mode 100644 > drivers/net/af_xdp/rte_eth_af_xdp.c > create mode 100644 drivers/net/af_xdp/rte_pmd_af_xdp_version.map >=20 > -- > 2.13.6