From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB80FA2E1B for ; Mon, 2 Sep 2019 16:45:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9B0221E9C2; Mon, 2 Sep 2019 16:45:07 +0200 (CEST) Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) by dpdk.org (Postfix) with ESMTP id A16531E9BD for ; Mon, 2 Sep 2019 16:45:06 +0200 (CEST) Received: by mail-qk1-f195.google.com with SMTP id s18so1662925qkj.3 for ; Mon, 02 Sep 2019 07:45:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=ze1+LcmNHxm+mhIWlO0rUvVDvTiKJJw5XpNP8cCWkaE=; b=vS+PVtlvNNPV6M4Hk4EE1vNAmclYteuL3KM50OQXpQENteTdY0h3G5K5JSBq/k+oS1 gvyQ5xIPrMZVPSubIT5nUSoPswmt5TsLULj183XoaR9dzaoFIFEBfYn3RIu+x+zDQJOo XCrjo/QyjtczmxmA5Tfeh3buYOjPt9P/LJOt+gMZZsWvlWepvZcV/hLKZl/jIoTKRS3M 2f1Oe0GrUg3WmWPmd1XHXCak1OvgvOR/6kyUTksx4tZISIrElI6UbxYNOTmmssVCobPC f9Tm4gwJuo3INsUtoncY88mEdKvxfshMkOFuLrfuuKnTGucxIvmhXDTA0qamdpbUEK/d ZsaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ze1+LcmNHxm+mhIWlO0rUvVDvTiKJJw5XpNP8cCWkaE=; b=qbJ6wpcFnQVkoeEgZO5wwl+YYpYdTGGJdaNUdhSuV2NcOUN7YEhJrd0CSWrvZXKBkS nxKh7labM1a6OHr54dTGYrr+Q1iSbKdgMVIoXLNGpe0VaKDtyl7sUl483SGfVZj3nlky wkDduOWd1pvPEWO4so/itIpNTjw1U8seZDMPSUp8sBw5lg2OAoDzlzxXADv4p6YiBd8X 3DKM6fHFSoXaLAVBDa+EGP18IOxYQUFzSDfPLCVBonk/XgqcbROD7qH8qG+wxYkSRf59 BNciXoDKN5rOh7MOGWA0mqVdP+sejHJpsFxG3H+EwPVQhY2n3gHoIqucalgZOnV2lTFq w52w== X-Gm-Message-State: APjAAAUONdi0IWjrzM8xfZ0sX60KNjX3hRjvtcESB4bYJ4jlXnHZfOpT /w7OiwQK9zP5VHqxMkFqP8KegpS+f6GTi5K/J1I= X-Google-Smtp-Source: APXvYqzCWFlH0Z7xO9mQve9hF2TmLBedeHiKbIqty5ocMLU/dud0oE6tUn03w8nNk1It9y1cFLXJ0uxgdyNOndM8uYU= X-Received: by 2002:a37:e306:: with SMTP id y6mr28772442qki.174.1567435505941; Mon, 02 Sep 2019 07:45:05 -0700 (PDT) MIME-Version: 1.0 References: <20190829150259.22206-1-ciara.loftus@intel.com> <74F120C019F4A64C9B78E802F6AD4CC2791F7BB8@IRSMSX106.ger.corp.intel.com> <74F120C019F4A64C9B78E802F6AD4CC2791F7BD0@IRSMSX106.ger.corp.intel.com> In-Reply-To: <74F120C019F4A64C9B78E802F6AD4CC2791F7BD0@IRSMSX106.ger.corp.intel.com> From: William Tu Date: Mon, 2 Sep 2019 07:44:28 -0700 Message-ID: To: "Loftus, Ciara" Cc: "dev@dpdk.org" , "Ye, Xiaolong" , "Richardson, Bruce" , "Laatz, Kevin" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH] net/af_xdp: enable support for unaligned umem chunks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Sep 2, 2019 at 1:55 AM Loftus, Ciara wrote= : > > > > Hi Ciara, > > > > > > I haven't tried this patch but have a question. > > > > > > On Thu, Aug 29, 2019 at 8:04 AM Ciara Loftus > > wrote: > > > > > > > > This patch enables the unaligned chunks feature for AF_XDP which > > > > allows chunks to be placed at arbitrary places in the umem, as > > > > opposed to them being required to be aligned to 2k. This allows for > > > > DPDK application mempools to be mapped directly into the umem and i= n > > > > turn enable zero copy transfer between umem and the PMD. > > > > > > > > This patch replaces the zero copy via external mbuf mechanism > > > > introduced in commit e9ff8bb71943 ("net/af_xdp: enable zero copy by > > > external mbuf"). > > > > The pmd_zero copy vdev argument is also removed as now the PMD will > > > > auto-detect presence of the unaligned chunks feature and enable it > > > > if so and otherwise fall back to copy mode if not detected. > > > > > > > > When enabled, this feature significantly improves single-core > > > > performance of the PMD. > > > > > > Why using unaligned chunk feature improve performance? > > > Existing external mbuf already has zero copy between umem and PMD, > > and > > > your patch also does the same thing. So the improvement is from > > > somewhere else? > > > > Hi William, > > > > Good question. > > The external mbuf way indeed has zero copy however there's some > > additional complexity in that path in the management of the buf_ring. > > > > For example on the fill/rx path, in the ext mbuf solution one must dequ= eue > > an addr from the buf_ring and add it to the fill queue, allocate an mbu= f for > > the external mbuf, get a pointer to the data @ addr and attach the exte= rnal > > mbuf. With the new solution, we allocate an mbuf from the mempool, deri= ve > > the addr from the mbuf itself and add it to the fill queue, and then on= rx we > > can simply cast the pointer to the data @ addr to an mbuf and return it= to the > > user. > > On tx/complete, instead of dequeuing from the buf_ring to get a valid a= ddr > > we can again just derive it from the mbuf itself. > > > > I've performed some testing to compare the old vs new zc and found that= for > > the case where the PMD and IRQs are pinned to separate cores the > > difference is ~-5%, but for single-core case where the PMD and IRQs are > > pinned to the same core (with the need_wakeup feature enabled), or when > > multiple PMDs are forwarding to one another the difference is significa= nt. > > Please see below: > > > > ports queues/port pinning =CE=94 old zc > > 1 1 0 -4.74% > > 1 1 1 17.99% > > 2 1 0 -5.62% > > 2 1 1 71.77% > > 1 2 0 114.24% > > 1 2 1 134.88% > > Apologies, the last 4 figures above were comparing old memcpy vs zc. Corr= ected data set below: > > ports qs/port pinning =CE=94 old zc > 1 1 0 -4.74% > 1 1 1 17.99% > 2 1 0 -5.80% > 2 1 1 37.24% > 1 2 0 104.27% > 1 2 1 136.73% > > > > > FYI the series has been now merged into the bpf-next tree: > > https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf- > > next.git/commit/?id=3Dbdb15a29cc28f8155e20f7fb58b60ffc452f2d1b > > Great, thanks for sharing the number and explanation. William > > Thanks, > > Ciara > > > > > > > > Thank you > > > William > > > > > > > > > > > Signed-off-by: Ciara Loftus > > > > Signed-off-by: Kevin Laatz > > > > --- > > > > doc/guides/nics/af_xdp.rst | 1 - > > > > doc/guides/rel_notes/release_19_11.rst | 9 + > > > > drivers/net/af_xdp/rte_eth_af_xdp.c | 304 ++++++++++++++++++---= - > > -- > > > - > > > > 3 files changed, 231 insertions(+), 83 deletions(-) > > > > > > >