DPDK patches and discussions
 help / color / mirror / Atom feed
From: Xueming Li <xuemingl@mellanox.com>
To: Wu@dev-r630-06.mtbc.labs.mlnx, Jingjing <jingjing.wu@intel.com>,
	VanHaaren@dev-r630-06.mtbc.labs.mlnx,
	Harry <harry.van.haaren@intel.com>
Cc: Xueming Li <xuemingl@mellanox.com>, dev@dpdk.org
Subject: [dpdk-dev] [RFC v1 00/11] scapy/python extension
Date: Tue,  5 Dec 2017 12:48:27 +0800	[thread overview]
Message-ID: <20171205044838.21401-1-xuemingl@mellanox.com> (raw)
In-Reply-To: <20171019140649.26668-2-xuemingl@mellanox.com>

This patch introduces a set of new elements into DPDK:
1. python/scapy as a standard dpdk library, disabled by default.
   Could be used by any application to integrate powerful python features.
2. tx, rx and expect, blocking mode testpmd CLI command
   tx command send packets with scapy syntax template, support range/enum, be able to generate multiple flows in one command and DPDK speed.
   rx command receive packets with timeout, with several choices on how to dump packets and mbuf headers.
   expect command try to send out packet, receive and compare with the packet sent, support mbuf header and packet header comparation, show detail info or hexdiff if anything different. Expect command is suitable for unit test and regression test by saving batch expect scripts into a file and use 'load' command to invoke them together.
3. pktgen - new testpmd forwarding engine
   Used to support tx, rx and expect blocking command.
   Also support rxonly/loopback/forward/macswap non-blocking idle modes with several packet dump choice.
4. py command - new testpmd CLI
   Run and evaluate python clause in a global context, samples:
    - py a = 123; b=0xaaa; hex(a+b)
    - py eth=Ether();ip=IP();
    - py hexdump(eth/ip/UDP());
    - tx 0 eth/ip
   py shell: sneak into python shell

The purpose of this scapy/python integration is to help programmers to speed up unit test w/o writing complex c code, verify features with batch expect scripts. Testpmd already a widely used platform for people to setup PMD and verify features with rich CLI command, that's why I choose to make new CLI on this familier tool instead of creating a new one.

I made hundreds of expect test cases when developing rte_flow vxlan and GRE features, that makes me pretty confident to touch any code, hope this tool help you as well.

quick guide document:
	https://github.com/steevenlee/dpdk/blob/master_scapy/doc/guides/howto/scapy.rst
github branch:
        https://github.com/steevenlee/dpdk/tree/master_scapy


v1:
* make scapy standard lib, default to disabled
* introduce pktgen forwarding engine for tx, rx and expect command
* py command to evaluate python clause or invoke python shell in testpmd
* batch expect sample file

v0:
* basic scapy lib
* scapy command

Xueming Li (11):
  lib/cmdline: support backspace key
  lib/cmdline: init parse result memeory
  lib/cmdline: add echo support in batch loading from file
  app/testpmd: support command echo in CLI batch loading
  test: update batch loading test
  lib/python: add embedded python lib
  app/testpmd: add python command
  app/testpmd: add pktgen forwarding engine
  app/testpmd: add pktgen engine scapy commands
  test/expect: add expect test scripts
  doc/scapy: add scapy how-to guide

 app/test-pmd/Makefile                    |    6 +
 app/test-pmd/cmdline.c                   |   80 ++-
 app/test-pmd/pktgen.c                    | 1092 ++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c                   |    1 +
 app/test-pmd/testpmd.h                   |    5 +
 config/common_base                       |    6 +
 doc/guides/howto/scapy.rst               |  300 ++++++++
 lib/Makefile                             |    2 +
 lib/librte_cmdline/cmdline_parse.c       |    2 +
 lib/librte_cmdline/cmdline_rdline.c      |    1 +
 lib/librte_cmdline/cmdline_socket.c      |    5 +-
 lib/librte_cmdline/cmdline_socket.h      |    3 +-
 lib/librte_cmdline/cmdline_vt100.c       |    1 +
 lib/librte_cmdline/cmdline_vt100.h       |    1 +
 lib/librte_eal/common/include/rte_log.h  |    1 +
 lib/librte_python/Makefile               |   60 ++
 lib/librte_python/rte_python.c           |  387 +++++++++++
 lib/librte_python/rte_python.h           |   71 ++
 lib/librte_python/rte_python_version.map |   12 +
 mk/rte.app.mk                            |    1 +
 test/expect/init.exp                     |   28 +
 test/expect/rx.exp                       |  134 ++++
 test/test/test_cmdline_lib.c             |   10 +-
 23 files changed, 2199 insertions(+), 10 deletions(-)
 create mode 100644 app/test-pmd/pktgen.c
 create mode 100644 doc/guides/howto/scapy.rst
 create mode 100644 lib/librte_python/Makefile
 create mode 100644 lib/librte_python/rte_python.c
 create mode 100644 lib/librte_python/rte_python.h
 create mode 100644 lib/librte_python/rte_python_version.map
 create mode 100644 test/expect/init.exp
 create mode 100644 test/expect/rx.exp

-- 
2.13.3

  parent reply	other threads:[~2017-12-05  5:36 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-19 14:06 [dpdk-dev] [RFC PATCH 0/2] using scapy to generate packet templates Xueming Li
2017-10-19 14:06 ` [dpdk-dev] [RFC PATCH 1/2] app/testpmd: add packet template Xueming Li
2017-12-05  4:43   ` [dpdk-dev] [RFC v1 00/11] scapy/python extension Xueming Li
2017-12-05  4:45   ` Xueming Li
2017-12-05  4:48   ` Xueming Li [this message]
2017-12-05  4:55   ` Xueming Li
2017-12-05  6:14     ` Xueming(Steven) Li
2017-12-05  5:00   ` Xueming Li
2017-12-05  5:03   ` [dpdk-dev] [RFC v1 00/11] scappy/pythoon extension Xueming Li
2017-12-05  5:04   ` [dpdk-dev] [RFC v1 00/11] scapy/python extension Xueming Li
2017-12-10 23:16     ` Wiles, Keith
2019-01-10 13:06     ` Eelco Chaudron
2019-01-16 13:24       ` Xueming(Steven) Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 0/9] " Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 1/9] lib/cmdline: add echo support in batch loading from file Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 2/9] app/testpmd: support command echo in CLI batch loading Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 3/9] test: update batch loading test Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 4/9] lib/python: add embedded python lib Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 5/9] app/testpmd: add python command Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 6/9] app/testpmd: add pktgen forwarding engine Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 7/9] app/testpmd: add pktgen engine scapy commands Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 8/9] test/expect: add expect test scripts Xueming Li
2017-12-08  8:22   ` [dpdk-dev] [RFC v1 9/9] doc/scapy: add scapy how-to guide Xueming Li
2017-10-19 14:06 ` [dpdk-dev] [RFC PATCH 2/2] app/testpmd: add scapy command as pkt template Xueming Li
2017-10-19 15:21 ` [dpdk-dev] [RFC PATCH 0/2] using scapy to generate packet templates Van Haaren, Harry
2017-10-21 16:04   ` Xueming(Steven) Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171205044838.21401-1-xuemingl@mellanox.com \
    --to=xuemingl@mellanox.com \
    --cc=VanHaaren@dev-r630-06.mtbc.labs.mlnx \
    --cc=Wu@dev-r630-06.mtbc.labs.mlnx \
    --cc=dev@dpdk.org \
    --cc=harry.van.haaren@intel.com \
    --cc=jingjing.wu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).