DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v1 1/2] distributor lib performance enhancements
@ 2016-12-01  4:50 David Hunt
  2016-12-01  4:50 ` [dpdk-dev] [PATCH v1 1/2] lib: distributor " David Hunt
  2016-12-01  4:50 ` [dpdk-dev] [PATCH v1 2/2] example: distributor app modified to use burstAPI David Hunt
  0 siblings, 2 replies; 202+ messages in thread
From: David Hunt @ 2016-12-01  4:50 UTC (permalink / raw)
  To: dev; +Cc: bruce.richardson

This patch aims to improve the throughput of the distributor library.

It uses a similar handshake mechanism to the previous version of
the library, in that bits are used to indicate when packets are ready
to be sent to a worker and ready to be returned from a worker. One main
difference is that instead of sending one packet in a cache line, it makes
use of the 7 free spaces in the same cache line in order to send up to
8 packets at a time to/from a worker.

The flow matching algorithm has had significant re-work, and now keeps an
array of inflight flows and an array of backlog flows, and matches incoming
flows to the inflight/backlog flows of all workers so that flow pinning to
workers can be maintained.

Notes:
   Apps must now work in bursts, as up to 8 are given to a worker at a time
   For performance in matching, Flow ID's are 15-bits
   Original API (and code) is kept for backward compatibility

Performance Gains
   2.2GHz Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
   2 x XL710 40GbE NICS to 2 x 40Gbps traffic generator channels 64b packets
   separate cores for rx, tx, distributor
    1 worker  - 4.8x
    4 workers - 2.9x
    8 workers - 1.8x
   12 workers - 2.1x
   16 workers - 1.8x


[PATCH v1 1/2] lib: distributor performance enhancements
[PATCH v1 2/2] example: distributor app modified to use burstAPI

^ permalink raw reply	[flat|nested] 202+ messages in thread

end of thread, other threads:[~2017-03-29 14:48 UTC | newest]

Thread overview: 202+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-01  4:50 [dpdk-dev] [PATCH v1 1/2] distributor lib performance enhancements David Hunt
2016-12-01  4:50 ` [dpdk-dev] [PATCH v1 1/2] lib: distributor " David Hunt
2016-12-22  4:37   ` [dpdk-dev] [PATCH v2 0/5] distributor library " David Hunt
2016-12-22  4:37     ` [dpdk-dev] [PATCH v2 1/5] lib: distributor " David Hunt
2016-12-22 12:47       ` Jerin Jacob
2016-12-22 16:14         ` Hunt, David
2017-01-02 10:22       ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 0/6] distributor-performance-improvements David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 1/6] lib: distributor performance enhancements David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 2/6] lib: add distributor vector flow matching David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 3/6] test: unit tests for new distributor burst api David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 4/6] test: add distributor_perf autotest David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 5/6] example: distributor app showing burst api David Hunt
2017-01-02 10:22         ` [dpdk-dev] [WARNING: A/V UNSCANNABLE][PATCH v3 6/6] doc: distributor library changes for new " David Hunt
2017-01-09  7:50       ` [dpdk-dev] [PATCH v4 0/6] distributor library performance enhancements David Hunt
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 1/6] lib: distributor " David Hunt
2017-01-13 15:19           ` Bruce Richardson
2017-01-19 14:58             ` Hunt, David
2017-01-16 16:36           ` Bruce Richardson
2017-01-19 12:07             ` Hunt, David
2017-01-20  9:18           ` [dpdk-dev] [PATCH v5 0/6] distributor library " David Hunt
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 1/6] lib: distributor " David Hunt
2017-01-23  9:24               ` [dpdk-dev] [PATCH v6 0/6] distributor library " David Hunt
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 1/6] lib: distributor " David Hunt
2017-02-21  3:17                   ` [dpdk-dev] [PATCH v7 0/17] distributor library " David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 01/17] lib: rename legacy distributor lib files David Hunt
2017-02-21 10:27                       ` Hunt, David
2017-02-24 14:03                       ` Bruce Richardson
2017-03-01  9:55                         ` Hunt, David
2017-03-01  7:47                       ` [dpdk-dev] [PATCH v8 0/18] distributor library performance enhancements David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-06  9:10                           ` [dpdk-dev] [PATCH v9 00/18] distributor lib performance enhancements David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-15  6:19                               ` [dpdk-dev] [PATCH v10 0/18] distributor library performance enhancements David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-20 10:08                                   ` [dpdk-dev] [PATCH v11 0/18] distributor lib performance enhancements David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 02/18] lib: create private header file David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 03/18] lib: add new distributor code David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 04/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 05/18] test/distributor: extra params for autotests David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 06/18] lib: switch distributor over to new API David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 07/18] lib: make v20 header file private David Hunt
2017-03-27 13:10                                       ` Thomas Monjalon
2017-03-28  8:47                                         ` Hunt, David
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 08/18] lib: add symbol versioning to distributor David Hunt
2017-03-27 13:02                                       ` Thomas Monjalon
2017-03-28  8:25                                         ` Hunt, David
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 09/18] test: test single and burst distributor API David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 10/18] test: add perf test for distributor burst mode David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 11/18] examples/distributor: allow for extra stats David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 12/18] examples/distributor: wait for ports to come up David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 13/18] examples/distributor: add dedicated core for dist David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 14/18] examples/distributor: tweaks for performance David Hunt
2017-03-27 13:04                                       ` Thomas Monjalon
2017-03-28  8:45                                         ` Hunt, David
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 15/18] examples/distributor: give Rx thread a core David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 16/18] doc: distributor library changes for new burst API David Hunt
2017-03-24 14:49                                       ` Mcnamara, John
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 17/18] doc: distributor app " David Hunt
2017-03-20 10:08                                     ` [dpdk-dev] [PATCH v11 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-27 13:06                                     ` [dpdk-dev] [PATCH v11 0/18] distributor lib performance enhancements Thomas Monjalon
2017-03-29 14:48                                       ` Thomas Monjalon
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 02/18] lib: create private header file David Hunt
2017-03-15 17:18                                   ` Thomas Monjalon
2017-03-16 10:43                                     ` Hunt, David
2017-03-16 15:40                                       ` Thomas Monjalon
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 03/18] lib: add new distributor code David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 04/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 05/18] test/distributor: extra params for autotests David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 06/18] lib: switch distributor over to new API David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 07/18] lib: make v20 header file private David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 08/18] lib: add symbol versioning to distributor David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 09/18] test: test single and burst distributor API David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 10/18] test: add perf test for distributor burst mode David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 11/18] examples/distributor: allow for extra stats David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 12/18] examples/distributor: wait for ports to come up David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 13/18] examples/distributor: add dedicated core for dist David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 14/18] examples/distributor: tweaks for performance David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 15/18] examples/distributor: give Rx thread a core David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 16/18] doc: distributor library changes for new burst API David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 17/18] doc: distributor app " David Hunt
2017-03-15  6:19                                 ` [dpdk-dev] [PATCH v10 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 02/18] lib: create private header file David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 03/18] lib: add new burst oriented distributor structs David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 04/18] lib: add new distributor code David Hunt
2017-03-10 16:03                               ` Bruce Richardson
2017-03-14 10:43                                 ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 05/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 06/18] test/distributor: extra params for autotests David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 07/18] lib: switch distributor over to new API David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 08/18] lib: make v20 header file private David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 09/18] lib: add symbol versioning to distributor David Hunt
2017-03-10 16:22                               ` Bruce Richardson
2017-03-13 10:17                                 ` Hunt, David
2017-03-13 10:28                                 ` Hunt, David
2017-03-13 11:01                                   ` Van Haaren, Harry
2017-03-13 11:02                                     ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 10/18] test: test single and burst distributor API David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 11/18] test: add perf test for distributor burst mode David Hunt
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 12/18] examples/distributor: allow for extra stats David Hunt
2017-03-10 16:46                               ` Bruce Richardson
2017-03-14 10:44                                 ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 13/18] sample: distributor: wait for ports to come up David Hunt
2017-03-10 16:48                               ` Bruce Richardson
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 14/18] examples/distributor: give distributor a core David Hunt
2017-03-10 16:49                               ` Bruce Richardson
2017-03-14 10:48                                 ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 15/18] examples/distributor: limit number of Tx rings David Hunt
2017-03-10 16:50                               ` Bruce Richardson
2017-03-14 10:50                                 ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 16/18] examples/distributor: give Rx thread a core David Hunt
2017-03-10 16:51                               ` Bruce Richardson
2017-03-14  9:34                                 ` Hunt, David
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 17/18] doc: distributor library changes for new burst API David Hunt
2017-03-07 17:25                               ` Mcnamara, John
2017-03-06  9:10                             ` [dpdk-dev] [PATCH v9 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-10 16:54                             ` [dpdk-dev] [PATCH v9 00/18] distributor lib performance enhancements Bruce Richardson
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 02/18] lib: create private header file David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 03/18] lib: add new burst oriented distributor structs David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 04/18] lib: add new distributor code David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 05/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 06/18] test/distributor: extra params for autotests David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 07/18] lib: switch distributor over to new API David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 08/18] lib: make v20 header file private David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 09/18] lib: add symbol versioning to distributor David Hunt
2017-03-01 14:50                           ` Hunt, David
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 10/18] test: test single and burst distributor API David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 11/18] test: add perf test for distributor burst mode David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 12/18] examples/distributor: allow for extra stats David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 13/18] sample: distributor: wait for ports to come up David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 14/18] examples/distributor: give distributor a core David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 15/18] examples/distributor: limit number of Tx rings David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 16/18] examples/distributor: give Rx thread a core David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 17/18] doc: distributor library changes for new burst API David Hunt
2017-03-01  7:47                         ` [dpdk-dev] [PATCH v8 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 02/17] lib: symbol versioning of functions in distributor David Hunt
2017-02-24 14:05                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 03/17] lib: create rte_distributor_private.h David Hunt
2017-02-24 14:07                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 04/17] lib: add new burst oriented distributor structs David Hunt
2017-02-24 14:08                       ` Bruce Richardson
2017-03-01  9:57                         ` Hunt, David
2017-02-24 14:09                       ` Bruce Richardson
2017-03-01  9:58                         ` Hunt, David
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 05/17] lib: add new distributor code David Hunt
2017-02-24 14:11                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 06/17] lib: add SIMD flow matching to distributor David Hunt
2017-02-24 14:11                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 07/17] lib: apply symbol versioning to distibutor lib David Hunt
2017-02-21 11:50                       ` Hunt, David
2017-02-24 14:12                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 08/17] test: change params to distributor autotest David Hunt
2017-02-24 14:14                       ` Bruce Richardson
2017-03-01 10:06                         ` Hunt, David
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 09/17] test: switch distributor test over to burst API David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 10/17] test: test single and burst distributor API David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 11/17] test: add perf test for distributor burst mode David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 12/17] example: add extra stats to distributor sample David Hunt
2017-02-24 14:16                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 13/17] sample: distributor: wait for ports to come up David Hunt
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 14/17] sample: switch to new distributor API David Hunt
2017-02-24 14:16                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 15/17] lib: make v20 header file private David Hunt
2017-02-24 14:18                       ` Bruce Richardson
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 16/17] doc: distributor library changes for new burst api David Hunt
2017-02-21 16:18                       ` Mcnamara, John
2017-02-21  3:17                     ` [dpdk-dev] [PATCH v7 17/17] maintainers: add to distributor lib maintainers David Hunt
2017-02-24 14:01                     ` [dpdk-dev] [PATCH v7 0/17] distributor library performance enhancements Bruce Richardson
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 2/6] lib: add distributor vector flow matching David Hunt
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 3/6] test: unit tests for new distributor burst API David Hunt
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 4/6] test: add distributor perf autotest David Hunt
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 5/6] examples/distributor_app: showing burst API David Hunt
2017-01-23  9:24                 ` [dpdk-dev] [PATCH v6 6/6] doc: distributor library changes for new " David Hunt
2017-01-23 17:02                 ` [dpdk-dev] [PATCH v6 0/6] distributor library performance enhancements Bruce Richardson
2017-01-24  8:56                 ` Liu, Yong
2017-01-23 12:26               ` [dpdk-dev] [PATCH v5 1/6] lib: distributor " Bruce Richardson
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 2/6] lib: add distributor vector flow matching David Hunt
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 3/6] test: unit tests for new distributor burst API David Hunt
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 4/6] test: add distributor perf autotest David Hunt
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 5/6] examples/distributor_app: showing burst API David Hunt
2017-01-23 12:31               ` Bruce Richardson
2017-01-20  9:18             ` [dpdk-dev] [PATCH v5 6/6] doc: distributor library changes for new " David Hunt
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 2/6] lib: add distributor vector flow matching David Hunt
2017-01-13 15:26           ` Bruce Richardson
2017-01-19 14:59             ` Hunt, David
2017-01-16 16:40           ` Bruce Richardson
2017-01-19 12:11             ` Hunt, David
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 3/6] test: unit tests for new distributor burst api David Hunt
2017-01-13 15:33           ` Bruce Richardson
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 4/6] test: add distributor_perf autotest David Hunt
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 5/6] example: distributor app showing burst api David Hunt
2017-01-13 15:36           ` Bruce Richardson
2017-01-13 15:38           ` Bruce Richardson
2017-01-09  7:50         ` [dpdk-dev] [PATCH v4 6/6] doc: distributor library changes for new " David Hunt
2016-12-22  4:37     ` [dpdk-dev] [PATCH v2 2/5] test: unit tests for new distributor " David Hunt
2016-12-22  4:37     ` [dpdk-dev] [PATCH v2 3/5] test: add distributor_perf autotest David Hunt
2016-12-22 12:19       ` Jerin Jacob
2017-01-02 16:24         ` Hunt, David
2017-01-04 13:09           ` Jerin Jacob
2016-12-22  4:37     ` [dpdk-dev] [PATCH v2 4/5] example: distributor app showing burst api David Hunt
2016-12-22  4:37     ` [dpdk-dev] [PATCH v2 5/5] doc: distributor library changes for new " David Hunt
2016-12-01  4:50 ` [dpdk-dev] [PATCH v1 2/2] example: distributor app modified to use burstAPI David Hunt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).