* [dpdk-dev] [PATCH 1/4] test: make all lpm routes be of unsigned type
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
@ 2016-06-14 13:03 ` Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 2/4] test: change lpm routes file from header to data file Nikita Kozlov
` (4 subsequent siblings)
5 siblings, 0 replies; 21+ messages in thread
From: Nikita Kozlov @ 2016-06-14 13:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
The one route that was different to the others in the test_lpm_routes.h
file was the entry "{0, 8}" which was the only route without a "U" after
the IP part. Add in the extra "U" to that entry so that it can be used
as a check character when parsing routes manually.
Signed-off-by: Nikita Kozlov <nikita@elyzion.net>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm_routes.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/app/test/test_lpm_routes.h b/app/test/test_lpm_routes.h
index 023b0f9..1decfee 100644
--- a/app/test/test_lpm_routes.h
+++ b/app/test/test_lpm_routes.h
@@ -281825,7 +281825,7 @@ static const struct route_rule large_route_table[] =
{3536977920U, 17},
{3392992768U, 23},
{3341675008U, 23},
- {0, 8},
+ {0U, 8},
{3326316168U, 30},
{3326316108U, 30},
{3326316060U, 30},
--
2.8.1
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH 2/4] test: change lpm routes file from header to data file
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 1/4] test: make all lpm routes be of unsigned type Nikita Kozlov
@ 2016-06-14 13:03 ` Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 3/4] test: change lpm test to use routes as resource Nikita Kozlov
` (3 subsequent siblings)
5 siblings, 0 replies; 21+ messages in thread
From: Nikita Kozlov @ 2016-06-14 13:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Change the file extension of the test_lpm_routes file from .h to .dat.
This makes the lines-of-code counts for DPDK more realistic as they are not
affected by the huge counts from the lpm data.
Signed-off-by: Nikita Kozlov <nikita@elyzion.net>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/Makefile | 1 +
app/test/{test_lpm_routes.h => test_lpm_routes.dat} | 0
2 files changed, 1 insertion(+)
rename app/test/{test_lpm_routes.h => test_lpm_routes.dat} (100%)
diff --git a/app/test/Makefile b/app/test/Makefile
index 7e4d484..a3f317d 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -118,6 +118,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_functions.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_scaling.c
+$(eval $(call linked_resource,test_lpm_data,test_lpm_routes.dat))
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c
diff --git a/app/test/test_lpm_routes.h b/app/test/test_lpm_routes.dat
similarity index 100%
rename from app/test/test_lpm_routes.h
rename to app/test/test_lpm_routes.dat
--
2.8.1
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH 3/4] test: change lpm test to use routes as resource
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 1/4] test: make all lpm routes be of unsigned type Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 2/4] test: change lpm routes file from header to data file Nikita Kozlov
@ 2016-06-14 13:03 ` Nikita Kozlov
2016-06-14 13:03 ` [dpdk-dev] [PATCH 4/4] test: change lpm perf " Nikita Kozlov
` (2 subsequent siblings)
5 siblings, 0 replies; 21+ messages in thread
From: Nikita Kozlov @ 2016-06-14 13:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Change the lpm autotest to use the routes data from the resource data
stored in the binary rather than including it directly into the C file
as a C header. This speeds up compile and link time, without changing
the test results.
Signed-off-by: Nikita Kozlov <nikita@elyzion.net>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 72 insertions(+), 1 deletion(-)
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index f6930fb..de54720 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -34,13 +34,26 @@
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
+#include <string.h>
#include <rte_lpm.h>
+#include <rte_ip.h>
+#include <rte_malloc.h>
#include "test.h"
-#include "test_lpm_routes.h"
+#include "resource.h"
+
#include "test_xmmt_ops.h"
+REGISTER_LINKED_RESOURCE(test_lpm_data)
+
+struct route_rule {
+ uint32_t ip;
+ uint8_t depth;
+};
+static struct route_rule *large_route_table;
+static unsigned int num_route_entries;
+
#define TEST_LPM_ASSERT(cond) do { \
if (!(cond)) { \
printf("Error at line %d: \n", __LINE__); \
@@ -1217,6 +1230,62 @@ test17(void)
return PASS;
}
+static int
+load_large_route_table(void)
+{
+ const struct resource *r;
+ const char *lpm_data;
+
+ r = resource_find("test_lpm_data");
+ TEST_ASSERT_NOT_NULL(r, "No large lpm table data found");
+
+ /* the routing table size is going to be less than the size of the
+ * resource, since text extries are more verbose. Allocate this as
+ * the max size, and shrink the allocation later
+ */
+ large_route_table = rte_malloc(NULL, resource_size(r), 0);
+ if (large_route_table == NULL)
+ return -1;
+
+ /* parse the lpm table. All entries are of format:
+ * {IP-as-decimal-unsigned, depth}
+ * For example:
+ * {1234567U, 24},
+ * We use the "U" and "}" characters as format check characters,
+ * after parsing each number.
+ */
+ for (lpm_data = r->begin; lpm_data < r->end; lpm_data++) {
+ if (*lpm_data == '{') {
+ char *endptr;
+
+ lpm_data++;
+ large_route_table[num_route_entries].ip = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != 'U') {
+ if (num_route_entries > 0)
+ printf("Failed parse of %.*s\n", 12, lpm_data);
+ continue;
+ }
+
+ lpm_data = endptr + 2; /* skip U and , */
+ large_route_table[num_route_entries].depth = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != '}') {
+ if (num_route_entries > 0)
+ printf("Failed parse of %.*s\n",5, lpm_data);
+ continue;
+ }
+
+ num_route_entries++;
+ }
+ }
+
+ large_route_table = rte_realloc(large_route_table,
+ sizeof(large_route_table[0]) * num_route_entries, 0);
+ printf("Read %u route entries\n", num_route_entries);
+ return 0;
+}
+
/*
* Do all unit tests.
*/
@@ -1227,6 +1296,8 @@ test_lpm(void)
unsigned i;
int status, global_status = 0;
+ TEST_ASSERT_SUCCESS(load_large_route_table(), "Error loading lpm table");
+
for (i = 0; i < NUM_LPM_TESTS; i++) {
status = tests[i]();
if (status < 0) {
--
2.8.1
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH 4/4] test: change lpm perf test to use routes as resource
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
` (2 preceding siblings ...)
2016-06-14 13:03 ` [dpdk-dev] [PATCH 3/4] test: change lpm test to use routes as resource Nikita Kozlov
@ 2016-06-14 13:03 ` Nikita Kozlov
2016-06-14 14:25 ` [dpdk-dev] [PATCH 0/4] Convert lpm data from header to resource Thomas Monjalon
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
5 siblings, 0 replies; 21+ messages in thread
From: Nikita Kozlov @ 2016-06-14 13:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Change the lpm perf autotest to use the routes data from the resource data
stored in the binary rather than including it directly into the C file
as a C header. This speeds up compile and link time, without changing
the test results.
Signed-off-by: Nikita Kozlov <nikita@elyzion.net>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm_perf.c | 83 ++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 76 insertions(+), 7 deletions(-)
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index 41da811..f72250f 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -34,16 +34,28 @@
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
+#include <string.h>
#include <rte_cycles.h>
#include <rte_random.h>
#include <rte_branch_prediction.h>
#include <rte_lpm.h>
+#include <rte_ip.h>
+#include <rte_malloc.h>
#include "test.h"
-#include "test_lpm_routes.h"
+#include "resource.h"
#include "test_xmmt_ops.h"
+REGISTER_LINKED_RESOURCE(test_lpm_data)
+
+struct route_rule {
+ uint32_t ip;
+ uint8_t depth;
+};
+static struct route_rule *large_route_table;
+static unsigned int num_route_entries;
+
#define TEST_LPM_ASSERT(cond) do { \
if (!(cond)) { \
printf("Error at line %d: \n", __LINE__); \
@@ -80,6 +92,62 @@ print_route_distribution(const struct route_rule *table, uint32_t n)
}
static int
+load_large_route_table(void)
+{
+ const struct resource *r;
+ const char *lpm_data;
+
+ r = resource_find("test_lpm_data");
+ TEST_ASSERT_NOT_NULL(r, "No large lpm table data found");
+
+ /* the routing table size is going to be less than the size of the
+ * resource, since text extries are more verbose. Allocate this as
+ * the max size, and shrink the allocation later
+ */
+ large_route_table = rte_malloc(NULL, resource_size(r), 0);
+ if (large_route_table == NULL)
+ return -1;
+
+ /* parse the lpm table. All entries are of format:
+ * {IP-as-decimal-unsigned, depth}
+ * For example:
+ * {1234567U, 24},
+ * We use the "U" and "}" characters as format check characters,
+ * after parsing each number.
+ */
+ for (lpm_data = r->begin; lpm_data < r->end; lpm_data++) {
+ if (*lpm_data == '{') {
+ char *endptr;
+
+ lpm_data++;
+ large_route_table[num_route_entries].ip = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != 'U') {
+ if (num_route_entries > 0)
+ printf("Failed parse of %.*s\n", 12, lpm_data);
+ continue;
+ }
+
+ lpm_data = endptr + 2; /* skip U and , */
+ large_route_table[num_route_entries].depth = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != '}') {
+ if (num_route_entries > 0)
+ printf("Failed parse of %.*s\n",5, lpm_data);
+ continue;
+ }
+
+ num_route_entries++;
+ }
+ }
+
+ large_route_table = rte_realloc(large_route_table,
+ sizeof(large_route_table[0]) * num_route_entries, 0);
+ printf("Read %u route entries\n", num_route_entries);
+ return 0;
+}
+
+static int
test_lpm_perf(void)
{
struct rte_lpm *lpm = NULL;
@@ -97,9 +165,10 @@ test_lpm_perf(void)
rte_srand(rte_rdtsc());
- printf("No. routes = %u\n", (unsigned) NUM_ROUTE_ENTRIES);
+ TEST_ASSERT_SUCCESS(load_large_route_table(), "Error loading lpm table");
+ printf("No. routes = %u\n", (unsigned) num_route_entries);
- print_route_distribution(large_route_table, (uint32_t) NUM_ROUTE_ENTRIES);
+ print_route_distribution(large_route_table, (uint32_t) num_route_entries);
lpm = rte_lpm_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
@@ -107,7 +176,7 @@ test_lpm_perf(void)
/* Measue add. */
begin = rte_rdtsc();
- for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
+ for (i = 0; i < num_route_entries; i++) {
if (rte_lpm_add(lpm, large_route_table[i].ip,
large_route_table[i].depth, next_hop_add) == 0)
status++;
@@ -136,7 +205,7 @@ test_lpm_perf(void)
(unsigned) cache_line_counter, (unsigned) cache_line_counter * 64);
printf("Average LPM Add: %g cycles\n",
- (double)total_time / NUM_ROUTE_ENTRIES);
+ (double)total_time / num_route_entries);
/* Measure single Lookup */
total_time = 0;
@@ -225,7 +294,7 @@ test_lpm_perf(void)
status = 0;
begin = rte_rdtsc();
- for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
+ for (i = 0; i < num_route_entries; i++) {
/* rte_lpm_delete(lpm, ip, depth) */
status += rte_lpm_delete(lpm, large_route_table[i].ip,
large_route_table[i].depth);
@@ -234,7 +303,7 @@ test_lpm_perf(void)
total_time += rte_rdtsc() - begin;
printf("Average LPM Delete: %g cycles\n",
- (double)total_time / NUM_ROUTE_ENTRIES);
+ (double)total_time / num_route_entries);
rte_lpm_delete_all(lpm);
rte_lpm_free(lpm);
--
2.8.1
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [dpdk-dev] [PATCH 0/4] Convert lpm data from header to resource
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
` (3 preceding siblings ...)
2016-06-14 13:03 ` [dpdk-dev] [PATCH 4/4] test: change lpm perf " Nikita Kozlov
@ 2016-06-14 14:25 ` Thomas Monjalon
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
5 siblings, 0 replies; 21+ messages in thread
From: Thomas Monjalon @ 2016-06-14 14:25 UTC (permalink / raw)
To: Nikita Kozlov; +Cc: dev, bruce.richardson
2016-06-14 15:03, Nikita Kozlov:
> Re-submission of Bruce Richardson patch set:
> http://dpdk.org/dev/patchwork/patch/12484/
>
> This patchset is a rebase to the current head and use of
> strndupa (not posix) was replaced.
[...]
> Nikita Kozlov (4):
> test: make all lpm routes be of unsigned type
> test: change lpm routes file from header to data file
> test: change lpm test to use routes as resource
> test: change lpm perf test to use routes as resource
Please fix the authorship and add your Signed-off at the end of the
commits you have touched.
There are few checkpatch warnings. It complains about unnecessary
line continuations and some spacing.
Thanks
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data
2016-06-14 13:03 ` [dpdk-dev] [PATCH " Nikita Kozlov
` (4 preceding siblings ...)
2016-06-14 14:25 ` [dpdk-dev] [PATCH 0/4] Convert lpm data from header to resource Thomas Monjalon
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 1/5] test: fix unneeded routes table include from lpm test Bruce Richardson
` (4 more replies)
5 siblings, 5 replies; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
This patchset takes the existing lpm large routing table information and
converts it from a header file included at compile time to a resource linked
in. This improves things in two ways:
1. Improves DPDK build time
2. Removes approx 1 million lines of code from our LOC counts, as the header
file no longer counts as code, but more correctly as data.
Before and after approx code stats (with some additional patchsets applied)
are given below. Notice how the app folder has dropped from being the biggest
component with 6x the drivers code to second place with less code than the
drivers.
Future work is to repeat the same process for lpm6. Additional work for both
lpm and lpm6 is to remove the routing tables entirely and replace them with
auto-generated data that gives the roughly the same results. That would
shrink the overall size of DPDK releases, and is the desired end state.
---
This patchset is based directly on the original RFC submitted:
http://www.dpdk.org/ml/archives/dev/2016-May/038478.html.
Updates in this version compared to RFC:
* rebased on dpdk.org HEAD. The changes are now mostly to test_lpm_perf.c
rather than test_lpm.c
* removed use of strndupa() as suggest by Nikita Koslov
---
## Before:
SLOC Directory SLOC-by-Language (Sorted)
1237464 app ansic=1236324,python=1140
192107 drivers ansic=192107
108109 lib ansic=107968,python=141
65447 examples ansic=65312,sh=135
849 tools python=444,sh=405
799 scripts sh=799
77 doc python=75,sh=2
0 config (none)
0 mk (none)
0 pkg (none)
0 top_dir (none)
Totals grouped by language (dominant language first):
ansic: 1601711 (99.80%)
python: 1800 (0.11%)
sh: 1341 (0.08%)
-----------
## After
SLOC Directory SLOC-by-Language (Sorted)
192107 drivers ansic=192107
160646 app ansic=159506,python=1140
108109 lib ansic=107968,python=141
65447 examples ansic=65312,sh=135
849 tools python=444,sh=405
799 scripts sh=799
77 doc python=75,sh=2
0 config (none)
0 mk (none)
0 pkg (none)
0 top_dir (none)
Totals grouped by language (dominant language first):
ansic: 524893 (99.41%)
python: 1800 (0.34%)
sh: 1341 (0.25%)
generated using David A. Wheeler's 'SLOCCount'
Bruce Richardson (5):
test: fix unneeded routes table include from lpm test
test: make all lpm routes be of unsigned type
test: add lpm routes as a linked resource
test: change lpm test to use routes as resource
test: change lpm routes file from header to data file
app/test/Makefile | 1 +
app/test/test_lpm.c | 2 +-
app/test/test_lpm_perf.c | 78 +++++++++++++++++++++-
.../{test_lpm_routes.h => test_lpm_routes.dat} | 2 +-
4 files changed, 80 insertions(+), 3 deletions(-)
rename app/test/{test_lpm_routes.h => test_lpm_routes.dat} (99%)
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 1/5] test: fix unneeded routes table include from lpm test
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 2/5] test: make all lpm routes be of unsigned type Bruce Richardson
` (3 subsequent siblings)
4 siblings, 0 replies; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
The large routing table header file in only needed by the performance
tests and not by the functional tests to remove the include of
test_lpm_routes.h from test_lpm.c.
Fixes: 62dac65a3265 ("app/test: isolate lpm performance cases")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index f6930fb..8d1bae9 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -36,9 +36,9 @@
#include <stdlib.h>
#include <rte_lpm.h>
+#include <rte_ip.h>
#include "test.h"
-#include "test_lpm_routes.h"
#include "test_xmmt_ops.h"
#define TEST_LPM_ASSERT(cond) do { \
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 2/5] test: make all lpm routes be of unsigned type
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 1/5] test: fix unneeded routes table include from lpm test Bruce Richardson
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 3/5] test: add lpm routes as a linked resource Bruce Richardson
` (2 subsequent siblings)
4 siblings, 0 replies; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
The one route that was different to the others in the test_lpm_routes.h
file was the entry "{0, 8}" which was the only route without a "U" after
the IP part. Add in the extra "U" to that entry so that it can be used
as a check character when parsing routes manually.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm_routes.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/app/test/test_lpm_routes.h b/app/test/test_lpm_routes.h
index 023b0f9..1decfee 100644
--- a/app/test/test_lpm_routes.h
+++ b/app/test/test_lpm_routes.h
@@ -281825,7 +281825,7 @@ static const struct route_rule large_route_table[] =
{3536977920U, 17},
{3392992768U, 23},
{3341675008U, 23},
- {0, 8},
+ {0U, 8},
{3326316168U, 30},
{3326316108U, 30},
{3326316060U, 30},
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 3/5] test: add lpm routes as a linked resource
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 1/5] test: fix unneeded routes table include from lpm test Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 2/5] test: make all lpm routes be of unsigned type Bruce Richardson
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 4/5] test: change lpm test to use routes as resource Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file Bruce Richardson
4 siblings, 0 replies; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
Since we now have the ability to store resource data directly inside
the test binary, take the test_lpm_routes.h header file and make it a
test resource. Later commits will then switch the test C file over to use
this rather than including it directly.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/Makefile | 1 +
app/test/test_lpm_perf.c | 3 +++
2 files changed, 4 insertions(+)
diff --git a/app/test/Makefile b/app/test/Makefile
index 2de8c7a..a147db4 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -127,6 +127,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_functions.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_scaling.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_multiwriter.c
+$(eval $(call linked_resource,test_lpm_data,test_lpm_routes.h))
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index 41da811..0e64919 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -41,9 +41,12 @@
#include <rte_lpm.h>
#include "test.h"
+#include "resource.h"
#include "test_lpm_routes.h"
#include "test_xmmt_ops.h"
+REGISTER_LINKED_RESOURCE(test_lpm_data)
+
#define TEST_LPM_ASSERT(cond) do { \
if (!(cond)) { \
printf("Error at line %d: \n", __LINE__); \
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 4/5] test: change lpm test to use routes as resource
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
` (2 preceding siblings ...)
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 3/5] test: add lpm routes as a linked resource Bruce Richardson
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file Bruce Richardson
4 siblings, 0 replies; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
Change the lpm autotest to use the routes data from the resource data
stored in the binary rather than including it directly into the C file
as a C header. This speeds up compile and link time, without changing
the test results.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/test_lpm_perf.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 74 insertions(+), 1 deletion(-)
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index 0e64919..ad70b02 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -34,17 +34,25 @@
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
+#include <string.h>
#include <rte_cycles.h>
#include <rte_random.h>
+#include <rte_malloc.h>
#include <rte_branch_prediction.h>
#include <rte_lpm.h>
#include "test.h"
#include "resource.h"
-#include "test_lpm_routes.h"
#include "test_xmmt_ops.h"
+struct route_rule {
+ uint32_t ip;
+ uint8_t depth;
+};
+static struct route_rule *large_route_table;
+static unsigned int NUM_ROUTE_ENTRIES;
+
REGISTER_LINKED_RESOURCE(test_lpm_data)
#define TEST_LPM_ASSERT(cond) do { \
@@ -83,6 +91,68 @@ print_route_distribution(const struct route_rule *table, uint32_t n)
}
static int
+load_large_route_table(void)
+{
+ const struct resource *r;
+ const char *lpm_data;
+
+ r = resource_find("test_lpm_data");
+ TEST_ASSERT_NOT_NULL(r, "No large lpm table data found");
+
+ /* the routing table size is going to be less than the size of the
+ * resource, since text extries are more verbose. Allocate this as
+ * the max size, and shrink the allocation later
+ */
+ large_route_table = rte_malloc(NULL, resource_size(r), 0);
+ if (large_route_table == NULL)
+ return -1;
+
+ /* parse the lpm table. All entries are of format:
+ * {IP-as-decimal-unsigned, depth}
+ * For example:
+ * {1234567U, 24},
+ * We use the "U" and "}" characters as format check characters,
+ * after parsing each number.
+ */
+ for (lpm_data = r->begin; lpm_data < r->end; lpm_data++) {
+ if (*lpm_data == '{') {
+ char *endptr;
+
+ lpm_data++;
+ large_route_table[NUM_ROUTE_ENTRIES].ip = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != 'U') {
+ if (NUM_ROUTE_ENTRIES > 0) {
+ char *value = strndup(lpm_data, 12);
+ printf("Failed parse of %s\n", value);
+ free(value);
+ }
+ continue;
+ }
+
+ lpm_data = endptr + 2; /* skip U and , */
+ large_route_table[NUM_ROUTE_ENTRIES].depth = \
+ strtoul(lpm_data, &endptr, 0);
+ if (*endptr != '}') {
+ if (NUM_ROUTE_ENTRIES > 0) {
+ char *value = strndup(lpm_data, 5);
+ printf("Failed parse of %s\n", value);
+ free(value);
+ }
+ continue;
+ }
+
+ NUM_ROUTE_ENTRIES++;
+ }
+ }
+
+ large_route_table = rte_realloc(large_route_table,
+ sizeof(large_route_table[0]) * NUM_ROUTE_ENTRIES, 0);
+ printf("Read %u route entries\n", NUM_ROUTE_ENTRIES);
+ return 0;
+}
+
+static int
test_lpm_perf(void)
{
struct rte_lpm *lpm = NULL;
@@ -98,6 +168,9 @@ test_lpm_perf(void)
uint64_t cache_line_counter = 0;
int64_t count = 0;
+ TEST_ASSERT_SUCCESS(load_large_route_table(),
+ "Error loading lpm table");
+
rte_srand(rte_rdtsc());
printf("No. routes = %u\n", (unsigned) NUM_ROUTE_ENTRIES);
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 0/5] change lpm route table from code to data Bruce Richardson
` (3 preceding siblings ...)
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 4/5] test: change lpm test to use routes as resource Bruce Richardson
@ 2016-07-14 16:06 ` Bruce Richardson
2016-07-15 23:05 ` Thomas Monjalon
4 siblings, 1 reply; 21+ messages in thread
From: Bruce Richardson @ 2016-07-14 16:06 UTC (permalink / raw)
To: dev; +Cc: Nikita Kozlov, Thomas Monjalon, Bruce Richardson
Change the file extension of the test_lpm_routes file from .h to .dat.
This makes the lines-of-code counts for DPDK more realistic as they are not
affected by the huge counts from the lpm data.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/Makefile | 2 +-
app/test/{test_lpm_routes.h => test_lpm_routes.dat} | 0
2 files changed, 1 insertion(+), 1 deletion(-)
rename app/test/{test_lpm_routes.h => test_lpm_routes.dat} (100%)
diff --git a/app/test/Makefile b/app/test/Makefile
index a147db4..cc8c205 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -127,7 +127,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_functions.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_scaling.c
SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_multiwriter.c
-$(eval $(call linked_resource,test_lpm_data,test_lpm_routes.h))
+$(eval $(call linked_resource,test_lpm_data,test_lpm_routes.dat))
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c
diff --git a/app/test/test_lpm_routes.h b/app/test/test_lpm_routes.dat
similarity index 100%
rename from app/test/test_lpm_routes.h
rename to app/test/test_lpm_routes.dat
--
2.5.5
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file
2016-07-14 16:06 ` [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file Bruce Richardson
@ 2016-07-15 23:05 ` Thomas Monjalon
2016-07-16 9:46 ` Richardson, Bruce
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Monjalon @ 2016-07-15 23:05 UTC (permalink / raw)
To: Bruce Richardson, Nikita Kozlov; +Cc: dev
2016-07-14 17:06, Bruce Richardson:
> Change the file extension of the test_lpm_routes file from .h to .dat.
> This makes the lines-of-code counts for DPDK more realistic as they are not
> affected by the huge counts from the lpm data.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> app/test/Makefile | 2 +-
> app/test/{test_lpm_routes.h => test_lpm_routes.dat} | 0
The size of the git tree significantly increase after renaming this file:
git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0
-c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc
--aggressive && du -sk .git
28272 .git
git am
git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0
-c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc
--aggressive && du -sk .git
33188 .git
I think it would be better to replace this huge file by a script generating
the test resource.
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [dpdk-dev] [PATCH v2 5/5] test: change lpm routes file from header to data file
2016-07-15 23:05 ` Thomas Monjalon
@ 2016-07-16 9:46 ` Richardson, Bruce
0 siblings, 0 replies; 21+ messages in thread
From: Richardson, Bruce @ 2016-07-16 9:46 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Nikita Kozlov, dev
> On 16 Jul 2016, at 00:05, Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
>
> 2016-07-14 17:06, Bruce Richardson:
>> Change the file extension of the test_lpm_routes file from .h to .dat.
>> This makes the lines-of-code counts for DPDK more realistic as they are not
>> affected by the huge counts from the lpm data.
>>
>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>> ---
>> app/test/Makefile | 2 +-
>> app/test/{test_lpm_routes.h => test_lpm_routes.dat} | 0
>
> The size of the git tree significantly increase after renaming this file:
> git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0
> -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc
> --aggressive && du -sk .git
> 28272 .git
> git am
> git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0
> -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc
> --aggressive && du -sk .git
> 33188 .git
>
> I think it would be better to replace this huge file by a script generating
> the test resource.
>
Completely agree. This was a temporary fix anyway so if it causes git bloat lets just wait for the permanent fix.
Bruce
^ permalink raw reply [flat|nested] 21+ messages in thread