From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0091.outbound.protection.outlook.com [104.47.0.91]) by dpdk.org (Postfix) with ESMTP id 4C3455A6E for ; Mon, 4 Sep 2017 12:44:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=c9ohKzcWhwxC3e9RMXjSi5fwPqTYIjQjKQUndHUvwR8=; b=jAdWHk5rcmKEW1DfaOCaISCfvL8Slq/cexQXF/umy/Kz7UUU/EJaYQ15CZQTOrjMaeuXPxih7LG5CCjtvfbG/3ElX+VlWQ3FSY2uYSrZiIWMtJ5Eks35nYwLCHQd6tPViy7DwzjoNvHnJM6NwusMr35f4Xq91bsKzBYOAzfuxlA= Received: from HE1PR83MB0220.EURPRD83.prod.outlook.com (141.251.204.155) by HE1PR83MB0057.EURPRD83.prod.outlook.com (129.75.21.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.0; Mon, 4 Sep 2017 10:44:03 +0000 Received: from HE1PR83MB0220.EURPRD83.prod.outlook.com ([fe80::a0d0:629b:2986:c06b]) by HE1PR83MB0220.EURPRD83.prod.outlook.com ([fe80::a0d0:629b:2986:c06b%7]) with mapi id 15.20.0035.007; Mon, 4 Sep 2017 10:44:03 +0000 From: Andrew Bainbridge To: "users@dpdk.org" Thread-Topic: Low level understanding of mbufs needed. Thread-Index: AdMlalzkoaxlS4W7SK+nLF4G43iFdQ== Date: Mon, 4 Sep 2017 10:44:02 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2a01:110:8012:1010::31f] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR83MB0057; 6:Yo+ROqw0n1HDz3MAaHvpoD5D1ZYbWg5qLtq0kL4dASZBB7in/f39a4N/LRpWt+j8VssMzmQVzP4YeZN8hu8LsO6ByVqMsadbJ1dcMlwlVTdE8KfnfWG0mzA7tN+b4DSrWbnVkXAgK8htDFsKAaxSO36wH2niaaLnJwloOt1cEd9h6PLjJkShLEGl0+sQ8AcILWlU1KKuxoFhBFOEviEoQTRjkktEEcV+PtqSkZaAt95ewCnX279+O4LR+GoBGTdddVJryr6VrcN6XKWvxYicInRMAoawcyJbQAk7/lSxGMRgGv/WZbCqRMYGCvDPTFyWstGiCQxC4Gbmdlv6XdL5TA==; 5:W0dyoDoNbu5Wvu1Z/lWobrmOpjmP5ie5QJ7HkN6OWhX0aX7Zq8sH/aHFEA/B8GJQ/DeF4LTpz9V3qB16YLeui99YgHsidrQP97lPT07iIUWArryBSE9EnHTPjw5zic5KN6uPK2xrd3Jgfs9MF55YmkWo/BBkWq5+2voF8btgJKE=; 24:2Y8zm+yXY/VlwZJhN8GMAERejLIJhibxDQdhskDE0yJtvnF6Bn8Sdb/6lRH50HpsmX8p8To9UKeRbyDiB7PSt/+omyyMasTbC+KcojKS5RA=; 7:R/cSWNrLf53zL9UalosVP0QE+KydOjw6NaceXfoleQUFqqGOnFZj0UTPZBDf5czI0JSCI06PnpUcr3JXmorvFTp7U6g/w55F56px33z7HxNfPoD9Jml1Ud+h9FVPitJCvip2okKQ4BhXU82pUveBd/Xn29IJ3G3kF16uFECvFzmitCROoNZ4qHJPuZo6Ho1u65J4kn8gdFPpd+ug91idXDsULqS7baU0SWiNi72RkYc= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 4f76f7e9-06e7-4e6d-72ef-08d4f381dc01 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:HE1PR83MB0057; x-ms-traffictypediagnostic: HE1PR83MB0057: x-exchange-antispam-report-test: UriScan:(21748063052155); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(61425038)(6040450)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(6055026)(61426038)(61427038)(6041248)(20161123560025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123562025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:HE1PR83MB0057; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:HE1PR83MB0057; x-forefront-prvs: 0420213CCD x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39860400002)(47760400005)(189002)(199003)(74316002)(8990500004)(3280700002)(110136004)(2351001)(8676002)(2900100001)(6916009)(86612001)(2906002)(6506006)(33656002)(5005710100001)(5630700001)(105586002)(6436002)(106356001)(7736002)(10290500003)(2501003)(7696004)(5250100002)(9686003)(478600001)(101416001)(189998001)(5660300001)(14454004)(54896002)(6306002)(97736004)(5640700003)(86362001)(10090500001)(99286003)(53936002)(3660700001)(790700001)(102836003)(6116002)(8936002)(50986999)(68736007)(25786009)(81156014)(55016002)(1730700003)(81166006)(54356999); DIR:OUT; SFP:1102; SCL:1; SRVR:HE1PR83MB0057; H:HE1PR83MB0220.EURPRD83.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=andbain@microsoft.com; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Sep 2017 10:44:02.8741 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR83MB0057 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Low level understanding of mbufs needed. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 10:44:07 -0000 I'm a newbie. I want to learn more about to use mbufs to achieve the best t= hroughput. My application is something like a VPN server. In pseudo code: while 1: pkt =3D recv() if pkt.ip.daddr =3D=3D CLIENT: new_pkt =3D encap(pkt) else: new_pkt =3D decap(pkt) send(new_pkt) Where encap() prepends an IP and UDP header, and decap() does the opposite. Most of each packet I send is the same as one I just received. Is it possib= le to do the send without having to allocate a new mbuf and memcpy into it? I want to learn more about how the system works at the low level. My guess of how it works is that the NIC reads in a packet from the Etherne= t cable and writes it into its on-chip SRAM. Once it has enough data buffer= ed, or enough time has elapsed it does a PCIe write request to copy the dat= a into system RAM. The simplest scheme would be to have a single large circ= ular buffer in system RAM and for the packets to be written nose-to-tail in= to that buffer. Does DPDK do that? I guess not. I guess the supported cards all support sca= tter/gather, which AFAICT means the NICs are smart enough to understand an = array of pointers to buffers. So what then? I have many 1500 byte buffers allocated, and I give the NIC a= n array of pointers to those buffers. The NIC then "scatters" the input str= eam into these buffers, one packet per buffer. I guess the best scheme for my application would be if I could tell the NIC= to always leave 30 bytes or so of headroom on each packet, so that I can p= repend the extra headers in the encap case. Can I request that when I confi= gure the mbufs? If you can point me to some kind of tutorial or blog post that covers this = area, that would also be helpful. Thanks, Andrew