From: Owen Hilyard <ohilyard@iol.unh.edu>
To: Owen Hilyard <ohilyard@iol.unh.edu>,
dts@dpdk.org, "Tu, Lijuan" <lijuan.tu@intel.com>
Cc: Sarah Hall <shall@iol.unh.edu>,
"Ma, LihongX" <lihongx.ma@intel.com>,
Lincoln Lavoie <lylavoie@iol.unh.edu>,
"Chen, Zhaoyan" <zhaoyan.chen@intel.com>,
"Peng, Yuan" <yuan.peng@intel.com>
Subject: Re: [dts] [PATCH v2] rte flow: add flow test generator
Date: Fri, 23 Oct 2020 19:30:54 -0400 [thread overview]
Message-ID: <CAHx6DYBUtMpMauQf=pKsg1SrtJL7-Fo=TXYTHa0dPHOJ8rtDPg@mail.gmail.com> (raw)
In-Reply-To: <20201023232903.37387-8-ohilyard@iol.unh.edu>
[-- Attachment #1: Type: text/plain, Size: 9241 bytes --]
I'm not sure why there's a duplicate of patch 7 attached here, please
ignore it.
On Fri, Oct 23, 2020 at 7:29 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:
> This file contains the actual generation logic for tests.
>
> The actual implimentation of the generation logic makes heavy use of the
> functional programming features in python due to the need to allow delayed
> execution through the entire chain of items. This is accomplished using a
> mixture of generator functions and the aformentioned functional programming
> capabilities. This delayed execution allows a for a much greater number of
> flow rules to be generated without causing issues in memory, since at most
> the program is storing a few dozen stack frames, instead of potentially a
> few thousand flow rules. This allows the generation logic to, with the
> proper parameters, generate all possible flow rules (ignoring properties).
> This would normally result in 2.5 million flow rules needing to be resident
> in memory before any properties could be changed. The approach of fully
> fuzzing the parser was not taken because, by my estimate, it would result
> in more than 11,479,792,543,757,705,936,896 flow rules.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
> framework/flow/generator.py | 165 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 165 insertions(+)
> create mode 100644 framework/flow/generator.py
>
> diff --git a/framework/flow/generator.py b/framework/flow/generator.py
> new file mode 100644
> index 0000000..0fe52b2
> --- /dev/null
> +++ b/framework/flow/generator.py
> @@ -0,0 +1,165 @@
> +# BSD LICENSE
> +#
> +# Copyright(c) 2020 Intel Corporation. All rights reserved.
> +# Copyright © 2018[, 2019] The University of New Hampshire. All rights
> reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +# * Redistributions of source code must retain the above copyright
> +# notice, this list of conditions and the following disclaimer.
> +# * Redistributions in binary form must reproduce the above copyright
> +# notice, this list of conditions and the following disclaimer in
> +# the documentation and/or other materials provided with the
> +# distribution.
> +# * Neither the name of Intel Corporation nor the names of its
> +# contributors may be used to endorse or promote products derived
> +# from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +from __future__ import annotations
> +
> +import copy
> +import itertools
> +import os
> +import sys
> +import time
> +from typing import List, Set, Generator, Iterable, FrozenSet, Tuple
> +
> +import numpy as np
> +
> +from flow.flow import Flow
> +from flow.flow_pattern_items import PATTERN_ITEMS_TYPE_CLASS_MAPPING,
> PatternFlowItem, \
> + PATTERN_OPERATION_TYPES, TUNNELING_PROTOCOL_TYPES,
> ALWAYS_ALLOWED_ITEMS, FlowItemEnd, FlowItemVxlan, FlowItemIpv4, \
> + FlowItemEth, FlowItemGre, L3_FLOW_TYPES, FlowItemVlan, FlowItemUdp
> +from flow.flow_rule import FlowItemType
> +
> +
> +def get_valid_next_protocols(current_protocol, protocol_stack,
> type_denylist):
> + return list(filter(
> + lambda patent_item: patent_item not in type_denylist and
> patent_item not in {p.type for p in protocol_stack},
> + current_protocol.valid_parent_items))
> +
> +
> +def _generate(type_denylist=None) -> List[List[PatternFlowItem]]:
> + if type_denylist is None:
> + type_denylist = set()
> + UNUSED_PATTERN_ITEMS = {PATTERN_ITEMS_TYPE_CLASS_MAPPING[i] for i in
> type_denylist}
> +
> + patterns: List[List[PatternFlowItem]] = []
> + for pattern_item in [clazz for clazz in
> PATTERN_ITEMS_TYPE_CLASS_MAPPING.values() if
> + clazz not in UNUSED_PATTERN_ITEMS]:
> + protocol_stack = []
> + if protocol_stack.count(pattern_item) >= 2:
> + continue
> +
> + current_protocol = pattern_item()
> + valid_next_protocols = get_valid_next_protocols(current_protocol,
> protocol_stack, type_denylist)
> + while len(valid_next_protocols) > 0:
> + protocol_stack.append(current_protocol)
> + current_protocol =
> PATTERN_ITEMS_TYPE_CLASS_MAPPING[list(valid_next_protocols)[0]]()
> + valid_next_protocols =
> get_valid_next_protocols(current_protocol, protocol_stack, type_denylist)
> +
> + protocol_stack.append(current_protocol)
> +
> + patterns.append(list(reversed(protocol_stack))) # This will
> place the lowest level protocols first
> + return patterns
> +
> +
> +def convert_protocol_stack_to_flow_pattern(protocol_stack):
> + return Flow(pattern_items=protocol_stack)
> +
> +
> +def _get_patterns_with_type_denylist(type_denylist: Set):
> + return [convert_protocol_stack_to_flow_pattern(protocol_stack) for
> protocol_stack in (_generate(
> + type_denylist=type_denylist
> + ))]
> +
> +
> +def _get_normal_protocol_patterns() -> List[Flow]:
> + return _get_patterns_with_type_denylist(
> + PATTERN_OPERATION_TYPES | ALWAYS_ALLOWED_ITEMS |
> {FlowItemType.ANY,
> +
> FlowItemType.END})
> +
> +
> +def _get_tunnelled_protocol_patterns(patterns: List[Flow]) ->
> Generator[Flow]:
> + VXLAN_FLOW = Flow(pattern_items=[FlowItemEth(), FlowItemIpv4(),
> FlowItemUdp(), FlowItemVxlan()])
> + for pattern in patterns:
> + yield VXLAN_FLOW / pattern
> +
> + GRE_FLOW = Flow(pattern_items=[FlowItemEth(), FlowItemIpv4(),
> FlowItemGre()])
> + for pattern in patterns:
> + if len(pattern.pattern_items) >= 2:
> + if pattern.pattern_items[1].type in L3_FLOW_TYPES:
> + yield GRE_FLOW / pattern
> +
> +
> +def get_patterns() -> Iterable[Iterable[Flow]]:
> + patterns: List[Flow] = _get_normal_protocol_patterns()
> +
> + # The flow with only an ethernet header was a consequence of the
> + # generation algorithm, but isn't that useful to test since we can't
> + # create a failing case without getting each NIC to write arbitrary
> + # bytes over the link.
> + eth_only_flow = Flow(pattern_items=[FlowItemEth()])
> + patterns.remove(eth_only_flow)
> +
> + # tunnelled_patterns = _get_tunnelled_protocol_patterns(patterns)
> +
> + return patterns
> +
> +
> +def add_properties_to_patterns(patterns: Iterable[Flow]) ->
> Iterable[Tuple[Flow, FrozenSet[str], FrozenSet[str], str]]:
> + test_property_flow_iters = map(lambda f: f.get_test_property_flows(),
> patterns)
> + for iterator in test_property_flow_iters:
> + yield from iterator
> +
> +
> +def get_patterns_with_properties() -> Iterable[Tuple[Flow,
> FrozenSet[str], FrozenSet[str], str]]:
> + base_patterns = get_patterns()
> + return add_properties_to_patterns(base_patterns)
> +
> +
> +def create_test_function_strings(test_configurations:
> Iterable[Tuple[Flow, FrozenSet[str], FrozenSet[str], str]]) -> \
> + Iterable[str]:
> + """
> + This will break if the __str__ methods of frozenset ever changes or
> if % formatting syntax is removed.
> +
> + @param test_configurations: An iterable with test configurations to
> convert into test case strings.
> + @return: An iterable containing strings that are function parameters.
> + """
> + function_template = \
> + """
> +def test_%s(self):
> + self.do_test_with_queue_action("%s", %s, %s)
> + """
> + return map(lambda test_configuration: function_template % (
> + test_configuration[-1], test_configuration[0],
> test_configuration[1], test_configuration[2],),
> + test_configurations)
> +
> +
> +def main():
> + """
> + Run this file (python3 generator.py) from the flow directory to print
> + out the pattern functions which are normally automatically generated
> + and added to the RTE Flow test suite at runtime.
> + """
> + pattern_tests = list(get_patterns_with_properties())
> + pattern_functions = create_test_function_strings(pattern_tests)
> + print("\n".join(pattern_functions))
> +
> +
> +if __name__ == "__main__":
> + main()
> --
> 2.25.1
>
>
[-- Attachment #2: Type: text/html, Size: 10361 bytes --]
next prev parent reply other threads:[~2020-10-23 23:31 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-14 20:50 [dts] [PATCH] rte flow: added test suite and framework Owen Hilyard
2020-10-14 21:02 ` [dts] [PATCH] rte flow: fixing checkpatch issues Owen Hilyard
2020-10-22 8:53 ` [dts] [PATCH] rte flow: added test suite and framework Tu, Lijuan
2020-10-23 18:18 ` Owen Hilyard
2020-10-23 23:28 ` [dts] [PATCH v2 1/7] rte flow: add test plan and test suite Owen Hilyard
2020-10-23 23:28 ` [dts] [PATCH v2 2/7] rte flow: add supporting data structures Owen Hilyard
2020-10-23 23:28 ` [dts] [PATCH v2 3/7] rte flow: Add base flow type Owen Hilyard
2020-10-23 23:28 ` [dts] [PATCH v2 4/7] rte flow: add more specifc data structures Owen Hilyard
2020-10-23 23:29 ` [dts] [PATCH v2 5/7] rte flow: add action items Owen Hilyard
2020-10-23 23:29 ` [dts] [PATCH v2 6/7] rte flow: add pattern items Owen Hilyard
2020-10-23 23:29 ` [dts] [PATCH v2 7/7] rte flow: add flow test generator Owen Hilyard
2020-10-23 23:29 ` [dts] [PATCH v2] " Owen Hilyard
2020-10-23 23:30 ` Owen Hilyard [this message]
2020-10-27 5:07 ` [dts] [PATCH v2 1/7] rte flow: add test plan and test suite Tu, Lijuan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAHx6DYBUtMpMauQf=pKsg1SrtJL7-Fo=TXYTHa0dPHOJ8rtDPg@mail.gmail.com' \
--to=ohilyard@iol.unh.edu \
--cc=dts@dpdk.org \
--cc=lihongx.ma@intel.com \
--cc=lijuan.tu@intel.com \
--cc=lylavoie@iol.unh.edu \
--cc=shall@iol.unh.edu \
--cc=yuan.peng@intel.com \
--cc=zhaoyan.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).