DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ciara Loftus <ciara.loftus@intel.com>
To: dev@dpdk.org
Subject: [dpdk-dev] [PATCH RFC 0/3] AF_XDP Preferred Busy Polling
Date: Thu, 18 Feb 2021 09:23:04 +0000
Message-ID: <20210218092307.29575-1-ciara.loftus@intel.com> (raw)

Single-core performance of AF_XDP at high loads can be poor because
a heavily loaded NAPI context will never enter or allow for busy-polling.

1C testpmd rxonly (both IRQs and PMD on core 0):
./dpdk-testpmd -l 0-1 --vdev=net_af_xdp0,iface=eth0 -- --forward-mode=rxonly
0.088Mpps

In order to achieve decent performance at high loads, it is currently
recommended ensure the IRQs for the netdev and the core running the PMD are
different.

2C testpmd rxonly (IRQs on core 0, PMD on core 1):
./dpdk-testpmd -l 0-1 --vdev=net_af_xdp0,iface=eth0 --main-lcore=0 -- \
--forward-mode=rxonly
19.26Mpps

However using an extra core is of course not ideal. The SO_PREFER_BUSY_POLL
socket option was introduced in kernel v5.11 to help improve 1C performance.
See [1].

This series sets this socket option on xsks created with DPDK (ie. instances of
the AF_XDP PMD) unless explicitly disabled or not supported by the kernel. It
was chosen to be enabled by default in order to bring the AF_XDP PMD in line
with most other PMDs which execute on a single core.

The following system and netdev settings are recommended in conjunction with
busy polling:
echo 2 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs
echo 200000 | sudo tee /sys/class/net/eth0/gro_flush_timeout

With the RFC these must be manually configured, but for the v1 these may be
set from the PMD since the performance is tightly coupled with these settings.

Re-running the 1C test with busy polling support and the above settings:
./dpdk-testpmd -l 0-1 --vdev=net_af_xdp0,iface=eth0 -- --forward-mode=rxonly
10.45Mpps

A new vdev arg is introduced called 'busy_budget' whose default value is 64.
busy_budget is the value supplied to the kernel with the SO_BUSY_POLL_BUDGET
socket option and represents the busy-polling NAPI budget ie. the number of
packets the kernel will attempt to process in the netdev's NAPI context. If set
to 0, preferred busy polling is disabled.

To set the busy budget to 256:
./dpdk-testpmd --vdev=net_af_xdp0,iface=eth0,busy_budget=256
14.06Mpps

To set the busy budget to 512:
./dpdk-testpmd --vdev=net_af_xdp0,iface=eth0,busy_budget=512
14.32Mpps

To disable preferred busy polling:
./dpdk-testpmd --vdev=net_af_xdp0,iface=eth0,busy_budget=0

[1] https://lwn.net/Articles/837010/


Ciara Loftus (3):
  net/af_xdp: Increase max batch size to 512
  net/af_xdp: Use recvfrom() instead of poll()
  net/af_xdp: preferred busy polling

 doc/guides/nics/af_xdp.rst          | 38 +++++++++++-
 drivers/net/af_xdp/compat.h         | 13 ++++
 drivers/net/af_xdp/rte_eth_af_xdp.c | 95 ++++++++++++++++++++++-------
 3 files changed, 124 insertions(+), 22 deletions(-)

-- 
2.17.1


             reply	other threads:[~2021-02-18  9:53 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-18  9:23 Ciara Loftus [this message]
2021-02-18  9:23 ` [dpdk-dev] [PATCH RFC 1/3] net/af_xdp: Increase max batch size to 512 Ciara Loftus
2021-02-18  9:23 ` [dpdk-dev] [PATCH RFC 2/3] net/af_xdp: Use recvfrom() instead of poll() Ciara Loftus
2021-02-18  9:23 ` [dpdk-dev] [PATCH RFC 3/3] net/af_xdp: preferred busy polling Ciara Loftus
2021-02-24 11:18 ` [dpdk-dev] [PATCH 0/3] AF_XDP Preferred Busy Polling Ciara Loftus
2021-02-24 11:18   ` [dpdk-dev] [PATCH 1/3] net/af_xdp: Increase max batch size to 512 Ciara Loftus
2021-03-01 16:31     ` Ferruh Yigit
2021-03-03 15:07       ` Loftus, Ciara
2021-03-03 15:38         ` Ferruh Yigit
2021-02-24 11:18   ` [dpdk-dev] [PATCH 2/3] net/af_xdp: Use recvfrom() instead of poll() Ciara Loftus
2021-02-24 11:18   ` [dpdk-dev] [PATCH 3/3] net/af_xdp: preferred busy polling Ciara Loftus
2021-03-01 16:32     ` Ferruh Yigit
2021-03-04 12:26       ` Loftus, Ciara
2021-03-08 15:54         ` Loftus, Ciara
2021-03-09 10:19   ` [dpdk-dev] [PATCH v2 0/3] AF_XDP Preferred Busy Polling Ciara Loftus
2021-03-09 10:19     ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: allow bigger batch sizes Ciara Loftus
2021-03-09 16:33       ` Ferruh Yigit
2021-03-10  7:21         ` Loftus, Ciara
2021-03-09 10:19     ` [dpdk-dev] [PATCH v2 2/3] net/af_xdp: Use recvfrom() instead of poll() Ciara Loftus
2021-03-09 10:19     ` [dpdk-dev] [PATCH v2 3/3] net/af_xdp: preferred busy polling Ciara Loftus
2021-03-09 16:33       ` Ferruh Yigit
2021-03-10  7:55         ` Loftus, Ciara
2021-03-10  7:48     ` [dpdk-dev] [PATCH v3 0/3] AF_XDP Preferred Busy Polling Ciara Loftus
2021-03-10  7:48       ` [dpdk-dev] [PATCH v3 1/3] net/af_xdp: allow bigger batch sizes Ciara Loftus
2021-03-10  7:48       ` [dpdk-dev] [PATCH v3 2/3] net/af_xdp: Use recvfrom() instead of poll() Ciara Loftus
2021-03-10  7:48       ` [dpdk-dev] [PATCH v3 3/3] net/af_xdp: preferred busy polling Ciara Loftus
2021-03-10 17:50       ` [dpdk-dev] [PATCH v3 0/3] AF_XDP Preferred Busy Polling Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210218092307.29575-1-ciara.loftus@intel.com \
    --to=ciara.loftus@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git