From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f41.google.com (mail-lf0-f41.google.com [209.85.215.41]) by dpdk.org (Postfix) with ESMTP id BFF1A37B0 for ; Mon, 4 Sep 2017 13:04:14 +0200 (CEST) Received: by mail-lf0-f41.google.com with SMTP id i79so593668lfe.4 for ; Mon, 04 Sep 2017 04:04:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=obtsWUg52fTYRS4AFsSP2fgWRoIrQvu4n2N6BKqDRUc=; b=bUA13PP/MSdbAi+2k84QakNGwSjfi33UVWgwglPonE+nxzgiYiry4tcljbhxkYKZ+q QtuuiOKLNqeQmYwCdddi2GhlaIpHZLjLTI7K9OrXhHpSXFqF8w5tPuBcI+JsvEQsWaD8 xOAQlfEh/OQ11AVD+LNheKy4vfjjxo4dbK/xilpwBHxwL6dy2ym55MqkU66eztuW5kZ2 xORCp44s1K3PSRlp2GF5nLM1xxK7F4kOo/VlRTMV6p/vyehoMnxTZxzUVe+NTb/RGosC z3JCxshZsVw8zQbc6U0RVJKqfLp+IEx3+Yf07EjIn6JjFo0uHuZeLCqV5phzanCn7o1b pitA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=obtsWUg52fTYRS4AFsSP2fgWRoIrQvu4n2N6BKqDRUc=; b=ejyLgpzgjvzS4Z523rodiv6crIfDICNv2LcHsBrB3p+E29f79STtfH8rkn/HpMO6pi ycYKg2lqrXrNAnvqyu8IzO+M0vRHm81zXWa7A+W94i5Kt7AZskZ5JTa85a26LKLpZbyC +fmJ1fZGZbIzqLrlg9S1OczITHsD63GvNYunPIyvjL8GS07eJStg2W/w6sg8LX2OfvZ1 SLY9/Wg90y2IV1dY5ZPG22CGlprSWE2f1qzQzhg30ljTc84XIdnxVYmzjwJwIBsDt9UD IOkwhnlVYIAg3xNxzK7RusPDDy6HgOTzcNZE9PkEoHwZgvmHvENrAyIttTfTsYzGHhuJ DoFQ== X-Gm-Message-State: AHPjjUj+9RBMIuKRfS5ZfvdEZA5iuKupSH/A1ttzdd1Q/sAIEPlG6Rng 6mr12biOKaEbavBvdFYhfrZ1xkbRjzA3 X-Google-Smtp-Source: ADKCNb582lzwosCLxYxRyFHZrnhOuIiappTMPWHbjBetsC6AWNIXjEhQlEUE19LAO7XPdGqeMK3HHt7fXZ7eJzCHw3A= X-Received: by 10.46.34.131 with SMTP id i125mr57204lji.34.1504523054403; Mon, 04 Sep 2017 04:04:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.179.0.8 with HTTP; Mon, 4 Sep 2017 04:03:53 -0700 (PDT) In-Reply-To: References: From: Andriy Berestovskyy Date: Mon, 4 Sep 2017 13:03:53 +0200 Message-ID: To: Andrew Bainbridge Cc: "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-users] Low level understanding of mbufs needed. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 11:04:14 -0000 Hey Andrew, Please note that each buffer has a headroom and a tailroom, so basically you might just encapsulate and decapsulate mbufs in-place, just modifying the headers and the adjusting the mbuf headroom. For general description please have a look here: http://dpdk.org/doc/guides/prog_guide/mbuf_lib.html The function you might want to use is rte_pktmbuf_adj(): http://dpdk.org/doc/api/rte__mbuf_8h.html#a1bbd752194759ce7b419c4998f2e8651 and rte_pktmbuf_prepend(): http://dpdk.org/doc/api/rte__mbuf_8h.html#aadf5bef4ceb0b76dfafff0895f285ab0 Regards, Andriy On Mon, Sep 4, 2017 at 12:44 PM, Andrew Bainbridge wrote: > I'm a newbie. I want to learn more about to use mbufs to achieve the best= throughput. My application is something like a VPN server. In pseudo code: > > while 1: > pkt =3D recv() > if pkt.ip.daddr =3D=3D CLIENT: > new_pkt =3D encap(pkt) > else: > new_pkt =3D decap(pkt) > send(new_pkt) > > Where encap() prepends an IP and UDP header, and decap() does the opposit= e. > > Most of each packet I send is the same as one I just received. Is it poss= ible to do the send without having to allocate a new mbuf and memcpy into i= t? > > I want to learn more about how the system works at the low level. > > My guess of how it works is that the NIC reads in a packet from the Ether= net cable and writes it into its on-chip SRAM. Once it has enough data buff= ered, or enough time has elapsed it does a PCIe write request to copy the d= ata into system RAM. The simplest scheme would be to have a single large ci= rcular buffer in system RAM and for the packets to be written nose-to-tail = into that buffer. > Does DPDK do that? I guess not. I guess the supported cards all support s= catter/gather, which AFAICT means the NICs are smart enough to understand a= n array of pointers to buffers. > > So what then? I have many 1500 byte buffers allocated, and I give the NIC= an array of pointers to those buffers. The NIC then "scatters" the input s= tream into these buffers, one packet per buffer. > > I guess the best scheme for my application would be if I could tell the N= IC to always leave 30 bytes or so of headroom on each packet, so that I can= prepend the extra headers in the encap case. Can I request that when I con= figure the mbufs? > > If you can point me to some kind of tutorial or blog post that covers thi= s area, that would also be helpful. > > Thanks, > Andrew --=20 Andriy Berestovskyy