From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id D3404A10 for ; Wed, 15 Mar 2017 14:18:35 +0100 (CET) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP; 15 Mar 2017 06:18:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,168,1486454400"; d="scan'208";a="75672852" Received: from silpixa00397515.ir.intel.com (HELO silpixa00397515.ger.corp.intel.com) ([10.237.223.14]) by orsmga005.jf.intel.com with ESMTP; 15 Mar 2017 06:18:33 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com Date: Wed, 15 Mar 2017 06:19:09 +0000 Message-Id: <1489558767-56329-1-git-send-email-david.hunt@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1488791433-186137-2-git-send-email-david.hunt@intel.com> References: <1488791433-186137-2-git-send-email-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v10 0/18] distributor library performance enhancements X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Mar 2017 13:18:37 -0000 This patch aims to improve the throughput of the distributor library. It uses a similar handshake mechanism to the previous version of the library, in that bits are used to indicate when packets are ready to be sent to a worker and ready to be returned from a worker. One main difference is that instead of sending one packet in a cache line, it makes use of the 7 free spaces in the same cache line in order to send up to 8 packets at a time to/from a worker. The flow matching algorithm has had significant re-work, and now keeps an array of inflight flows and an array of backlog flows, and matches incoming flows to the inflight/backlog flows of all workers so that flow pinning to workers can be maintained. The Flow Match algorithm has both scalar and a vector versions, and a function pointer is used to select the post appropriate function at run time, depending on the presence of the SSE2 cpu flag. On non-x86 platforms, the the scalar match function is selected, which should still gives a good boost in performance over the non-burst API. v10 changes: * Addressed all review comments from v9 (thanks, Bruce) * Squashed the two patches containing distributor structs and code * Renamed confusing rte_distributor_v1705.h to rte_distributor_next.h * Added usleep in main so as to be a little more gentle with that core * Fixed some patch titles and improved some descriptions * Updated sample app guide documentation * Removed un-needed code limiting Tx rings and cleaned up patch * Inherited v9 series Ack by Bruce, except new suggested addition for example app documentation (17/18) v9 changes: * fixed symbol versioning so it will compile on CentOS and RedHat v8 changes: * Changed the patch set to have a more logical order order of the changes, but the end result is basically the same. * Fixed broken shared library build. * Split down the updates to example app more * No longer changes the test app and sample app to use a temporary API. * No longer temporarily re-names the functions in the version.map file. v7 changes: * Reorganised patch so there's a more natural progression in the changes, and divided them down into easier to review chunks. * Previous versions of this patch set were effectively two APIs. We now have a single API. Legacy functionality can be used by by using the rte_distributor_create API call with the RTE_DISTRIBUTOR_SINGLE flag when creating a distributor instance. * Added symbol versioning for old API so that ABI is preserved. v6 changes: * Fixed intermittent segfault where num pkts not divisible by BURST_SIZE * Cleanup due to review comments on mailing list * Renamed _priv.h to _private.h. v5 changes: * Removed some un-needed code around retries in worker API calls * Cleanup due to review comments on mailing list * Cleanup of non-x86 platform compilation, fallback to scalar match v4 changes: * fixed issue building shared libraries v3 changes: * Addressed mailing list review comments * Test code removal * Split out SSE match into separate file to facilitate NEON addition * Cleaned up conditional compilation flags for SSE2 * Addressed c99 style compilation errors * rebased on latest head (Jan 2 2017, Happy New Year to all) v2 changes: * Created a common distributor_priv.h header file with common definitions and structures. * Added a scalar version so it can be built and used on machines without sse2 instruction set * Added unit autotests * Added perf autotest Notes: Apps must now work in bursts, as up to 8 are given to a worker at a time For performance in matching, Flow ID's are 15-bits If 32 bits Flow IDs are required, use the packet-at-a-time (SINGLE) mode. Performance Gains 2.2GHz Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz 2 x XL710 40GbE NICS to 2 x 40Gbps traffic generator channels 64b packets separate cores for rx, tx, distributor 1 worker - up to 4.8x 4 workers - up to 2.9x 8 workers - up to 1.8x 12 workers - up to 2.1x 16 workers - up to 1.8x [01/18] lib: rename legacy distributor lib files [02/18] lib: create private header file [03/18] lib: add new distributor code [04/18] lib: add SIMD flow matching to distributor [05/18] test/distributor: extra params for autotests [06/18] lib: switch distributor over to new API [07/18] lib: make v20 header file private [08/18] lib: add symbol versioning to distributor [09/18] test: test single and burst distributor API [10/18] test: add perf test for distributor burst mode [11/18] examples/distributor: allow for extra stats [12/18] examples/distributor: wait for ports to come up [13/18] examples/distributor: add dedicated core for dist [14/18] examples/distributor: tweaks for performance [15/18] examples/distributor: give Rx thread a core [16/18] doc: distributor library changes for new burst API [17/18] doc: distributor app changes for new burst API [18/18] maintainers: add to distributor lib maintainers