From: "Loftus, Ciara" <ciara.loftus@intel.com>
To: William Tu <u9012063@gmail.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"Ye, Xiaolong" <xiaolong.ye@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"Laatz, Kevin" <kevin.laatz@intel.com>
Subject: Re: [dpdk-dev] [PATCH] net/af_xdp: enable support for unaligned umem chunks
Date: Mon, 2 Sep 2019 08:48:11 +0000 [thread overview]
Message-ID: <74F120C019F4A64C9B78E802F6AD4CC2791F7BB8@IRSMSX106.ger.corp.intel.com> (raw)
In-Reply-To: <CALDO+SY-_-iSpywRt6bq6mA_5txymOxRekvt-cEMez7yNtg8ng@mail.gmail.com>
> Hi Ciara,
>
> I haven't tried this patch but have a question.
>
> On Thu, Aug 29, 2019 at 8:04 AM Ciara Loftus <ciara.loftus@intel.com> wrote:
> >
> > This patch enables the unaligned chunks feature for AF_XDP which
> > allows chunks to be placed at arbitrary places in the umem, as opposed
> > to them being required to be aligned to 2k. This allows for DPDK
> > application mempools to be mapped directly into the umem and in turn
> > enable zero copy transfer between umem and the PMD.
> >
> > This patch replaces the zero copy via external mbuf mechanism
> > introduced in commit e9ff8bb71943 ("net/af_xdp: enable zero copy by
> external mbuf").
> > The pmd_zero copy vdev argument is also removed as now the PMD will
> > auto-detect presence of the unaligned chunks feature and enable it if
> > so and otherwise fall back to copy mode if not detected.
> >
> > When enabled, this feature significantly improves single-core
> > performance of the PMD.
>
> Why using unaligned chunk feature improve performance?
> Existing external mbuf already has zero copy between umem and PMD, and
> your patch also does the same thing. So the improvement is from
> somewhere else?
Hi William,
Good question.
The external mbuf way indeed has zero copy however there's some additional complexity in that path in the management of the buf_ring.
For example on the fill/rx path, in the ext mbuf solution one must dequeue an addr from the buf_ring and add it to the fill queue, allocate an mbuf for the external mbuf, get a pointer to the data @ addr and attach the external mbuf. With the new solution, we allocate an mbuf from the mempool, derive the addr from the mbuf itself and add it to the fill queue, and then on rx we can simply cast the pointer to the data @ addr to an mbuf and return it to the user.
On tx/complete, instead of dequeuing from the buf_ring to get a valid addr we can again just derive it from the mbuf itself.
I've performed some testing to compare the old vs new zc and found that for the case where the PMD and IRQs are pinned to separate cores the difference is ~-5%, but for single-core case where the PMD and IRQs are pinned to the same core (with the need_wakeup feature enabled), or when multiple PMDs are forwarding to one another the difference is significant. Please see below:
ports queues/port pinning Δ old zc
1 1 0 -4.74%
1 1 1 17.99%
2 1 0 -5.62%
2 1 1 71.77%
1 2 0 114.24%
1 2 1 134.88%
FYI the series has been now merged into the bpf-next tree:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=bdb15a29cc28f8155e20f7fb58b60ffc452f2d1b
Thanks,
Ciara
>
> Thank you
> William
>
> >
> > Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
> > Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
> > ---
> > doc/guides/nics/af_xdp.rst | 1 -
> > doc/guides/rel_notes/release_19_11.rst | 9 +
> > drivers/net/af_xdp/rte_eth_af_xdp.c | 304 ++++++++++++++++++------
> -
> > 3 files changed, 231 insertions(+), 83 deletions(-)
> >
> <snip>
next prev parent reply other threads:[~2019-09-02 8:48 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-29 15:02 Ciara Loftus
2019-08-30 7:47 ` Loftus, Ciara
2019-08-30 16:07 ` William Tu
2019-09-02 8:48 ` Loftus, Ciara [this message]
2019-09-02 8:55 ` Loftus, Ciara
2019-09-02 14:44 ` William Tu
2019-09-03 22:02 ` Ye Xiaolong
2019-09-04 10:30 ` Loftus, Ciara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=74F120C019F4A64C9B78E802F6AD4CC2791F7BB8@IRSMSX106.ger.corp.intel.com \
--to=ciara.loftus@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=kevin.laatz@intel.com \
--cc=u9012063@gmail.com \
--cc=xiaolong.ye@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).