DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [RFC] vhost packed ring performance optimization
@ 2019-07-08 17:13 Marvin Liu
  2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 01/13] add vhost normal enqueue function Marvin Liu
                   ` (12 more replies)
  0 siblings, 13 replies; 25+ messages in thread
From: Marvin Liu @ 2019-07-08 17:13 UTC (permalink / raw)
  To: tiwei.bie, maxime.coquelin, dev; +Cc: Marvin Liu

Packed ring has more compact ring format and thus can significantly
reduced the number of cache miss. It can lead to better performance.
This has been approved in virtio user driver, on normal E5 Xeon cpu
single core performance can raise 12%.

http://mails.dpdk.org/archives/dev/2018-April/095470.html

However vhost performance with packed ring performance was decreased.
Through analysis, mostly extra cost was from the calculating of each
descriptor flag which depended on ring wrap counter. Moreover, both
frontend and backend need to write same descriptors which will cause
cache contention. Especially when doing vhost enqueue function, virtio
refill packed ring function may write same cache line when vhost doing
enqueue function. This kind of extra cache cost will neutralize the
benefit of reducing cache misses. 

For optimizing vhost packed ring performance, vhost enqueue and dequeue
function will be divided into fast and normal parts.

Several methods will be taken in fast path:
	Uroll burst loop function into more pieces.
	Handle descriptors in one cache line simultaneously.
	Prerequisite check that whether I/O space can copy directly
	into mbuf space and vice versa.
	Prerequisite check that whether descriptor mapping is
	successful.
	Distinguish vhost descriptor update function by enqueue and
	dequeue function.
	Buffer dequeue used descriptors as many as possible.
	Update enqueue used descriptors by cache line.
	Cached memory region structure for fast conversion.
	Defined macros for pre-calculating packed descriptors flag.

Indirect and merged packets will be handled in normal path, as most-likely
they are large packets and most of costs are in memory copy.

After all these methods done, single core vhost PvP performance with 64B
packet on Xeon 8180 can boost 35%, loopback performance measured by virtio
user pmd can boost over 45%.

Marvin Liu (13):
  add vhost normal enqueue function
  add vhost packed ring fast enqueue function
  add vhost packed ring normal dequeue function
  add vhost packed ring fast dequeue function
  add enqueue shadow used descs update and flush functions
  add vhost fast enqueue flush function
  add vhost dequeue shadow descs update function
  add vhost fast dequeue flush function
  replace vhost enqueue packed ring function
  add vhost fast zero copy dequeue packed ring function
  replace vhost dequeue packed ring function
  support inorder in vhost dequeue path
  remove useless vhost functions

 lib/librte_vhost/vhost.h      |   16 +
 lib/librte_vhost/virtio_net.c | 1048 +++++++++++++++++++++++++++------
 2 files changed, 883 insertions(+), 181 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2019-08-13  9:09 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-08 17:13 [dpdk-dev] [RFC] vhost packed ring performance optimization Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 01/13] add vhost normal enqueue function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 02/13] add vhost packed ring fast " Marvin Liu
     [not found]   ` <CGME20190708113801eucas1p25d89717d8b298790326077852c9933c8@eucas1p2.samsung.com>
2019-07-08 11:37     ` Ilya Maximets
2019-07-09  1:15       ` Liu, Yong
2019-07-10  4:28   ` Jason Wang
2019-07-10  7:30     ` Liu, Yong
2019-07-11  4:11       ` Jason Wang
2019-07-11  9:49         ` Liu, Yong
2019-07-11  9:54           ` Jason Wang
2019-08-13  9:02             ` Liu, Yong
2019-07-11  8:35   ` Jason Wang
2019-07-11  9:37     ` Liu, Yong
2019-07-11  9:44       ` Jason Wang
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 03/13] add vhost packed ring normal dequeue function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 04/13] add vhost packed ring fast " Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 05/13] add enqueue shadow used descs update and flush functions Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 06/13] add vhost fast enqueue flush function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 07/13] add vhost dequeue shadow descs update function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 08/13] add vhost fast dequeue flush function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 09/13] replace vhost enqueue packed ring function Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 10/13] add vhost fast zero copy dequeue " Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 11/13] replace vhost " Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 12/13] support inorder in vhost dequeue path Marvin Liu
2019-07-08 17:13 ` [dpdk-dev] [RFC PATCH 13/13] remove useless vhost functions Marvin Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).