Re: [dpdk-dev] [PATCH v6 00/10] examples/ipsec-secgw: make app to use ipsec library

[Akhil Goyal <akhil.goyal@nxp.com>]

On 1/4/2019 8:10 PM, Akhil Goyal wrote:
>
> On 1/4/2019 5:59 PM, Ananyev, Konstantin wrote:
>> Hi Akhil,
>>
>>> Hi Konstantin,
>>>
>>> With this patchset, I am seeing a 3% drop in legacy mode lookaside and
>>> 12% drop with -l option.
>>> I am debugging this. Will let you know if I find something.
>> Ok, thanks.
>> For legacy mode do you know which patch in the series causing 3% drop?
>> Is it still: " fix crypto-op might never get dequeued" or a different one?
>> Konstantin
>>
> for legacy mode you may consider this diff in 3/10
>
> diff --git a/examples/ipsec-secgw/ipsec-secgw.c
> b/examples/ipsec-secgw/ipsec-secgw.c
> index 4f7a77d8d..d183862b8 100644
> --- a/examples/ipsec-secgw/ipsec-secgw.c
> +++ b/examples/ipsec-secgw/ipsec-secgw.c
> @@ -1015,10 +1015,13 @@ main_loop(__attribute__((unused)) void *dummy)
>
>                           if (nb_rx > 0)
>                                   process_pkts(qconf, pkts, nb_rx, portid);
> -               }
> +//             }
>
> -               drain_inbound_crypto_queues(qconf, &qconf->inbound);
> -               drain_outbound_crypto_queues(qconf, &qconf->outbound);
> +                       if (UNPROTECTED_PORT(portid))
> +                               drain_inbound_crypto_queues(qconf,
> &qconf->inbound);
> +                       else
> +                               drain_outbound_crypto_queues(qconf,
> &qconf->outbound);
> +               }
>           }
>    }
>
> The 3% gap was on single core performance. But on multi cores scenario,
> there is no significant drop.
>
> But I see a bigger functional problem in case of non-legacy mode.
> I am trying a multi tunnel scenario with 32 kind of flows on one side
> each with different tunnels.
> The flows are kind of dest ip: 192.168.101.1 ........... 192.168.x.1
> .........  192.168.132.1.
> Each IP has a different tunnel.
> All are pumped from same port. I can see that some of the IPs are not
> getting forwarded.
> like 192.168.103.1, 192.168.104.1, 192.168.105.1, 192.168.114.1,
> 192.168.115.1, 192.168.116.1, 192.168.122.1, 192.168.125.1,
> 192.168.126.1, 192.168.130.1
>
> The same setup/flows works perfectly fine with legacy mode.

I see that the IPs are not fixed and changes on every run. Probably, the 
inbound /outbound queues gets flushed when they do not intend to on that 
particular core and packets get discarded as they come from non-destined 
port.
Also the issue does not come if I add the above suggested change.

>
>
>>> -Akhil
>>>
>>> On 1/4/2019 1:55 AM, Konstantin Ananyev wrote:
>>>> This patch series depends on the patch series:
>>>>
>>>> ipsec: new library for IPsec data-path processing
>>>> http://patches.dpdk.org/patch/49410/
>>>> http://patches.dpdk.org/patch/49411/
>>>> http://patches.dpdk.org/patch/49412/
>>>> http://patches.dpdk.org/patch/49413/
>>>> http://patches.dpdk.org/patch/49414/
>>>> http://patches.dpdk.org/patch/49415/
>>>> http://patches.dpdk.org/patch/49416/
>>>> http://patches.dpdk.org/patch/49417/
>>>> http://patches.dpdk.org/patch/49418/
>>>> http://patches.dpdk.org/patch/49419/
>>>>
>>>> to be applied first.
>>>>
>>>> v5 -> v6
>>>>      Address issues reported by Akhil:
>>>>         segfault when using lookaside-proto device
>>>>         HW IPv4 cksum offload not enabled by default
>>>>         crypto-dev dequeue() is called to often
>>>>
>>>> v4 -> v5
>>>> - Address Akhil comments:
>>>>         documentation update
>>>>         spell checks spacing etc.
>>>>         introduce rxoffload/txoffload parameters
>>>>         single SA for ipv6
>>>>         update Makefile
>>>>
>>>> v3 -> v4
>>>>     - fix few issues with the test scripts
>>>>     - update docs
>>>>
>>>> v2 -> v3
>>>>     - add IPv6 cases into test scripts
>>>>     - fixes for IPv6 support
>>>>     - fixes for inline-crypto support
>>>>     - some code restructuring
>>>>
>>>> v1 -> v2
>>>>     - Several bug fixes
>>>>
>>>> That series contians few bug-fixes and changes to make ipsec-secgw
>>>> to utilize librte_ipsec library:
>>>>         - changes in the related data structures.
>>>>         - changes in the initialization code.
>>>>         - changes in the data-path code.
>>>>         - new command-line parameters to enable librte_ipsec codepath
>>>>           and related features.
>>>>         - test scripts to help automate ipsec-secgw functional testing.
>>>>
>>>> Note that right now by default current (non-librte_ipsec) code-path
>>>> will be used. User has to run application with new command-line option
>>>> ('-l')
>>>> to enable new codepath.
>>>> The main reason for that:
>>>>      - current librte_ipsec doesn't support all ipsec algorithms
>>>>        and features that the app does.
>>>>      - allow users to run both versions in parallel for some time
>>>>        to figure out any functional or performance degradation with the
>>>>        new code.
>>>>
>>>> Test scripts were run with the following crypto devices:
>>>>     - aesni_mb
>>>>     - aesni_gcm
>>>>     - qat
>>>>
>>>> Konstantin Ananyev (10):
>>>>      examples/ipsec-secgw: allow user to disable some RX/TX offloads
>>>>      examples/ipsec-secgw: allow to specify neighbour mac address
>>>>      examples/ipsec-secgw: fix crypto-op might never get dequeued
>>>>      examples/ipsec-secgw: fix outbound codepath for single SA
>>>>      examples/ipsec-secgw: make local variables static
>>>>      examples/ipsec-secgw: fix inbound SA checking
>>>>      examples/ipsec-secgw: make app to use ipsec library
>>>>      examples/ipsec-secgw: make data-path to use ipsec library
>>>>      examples/ipsec-secgw: add scripts for functional test
>>>>      doc: update ipsec-secgw guide and relelase notes
>>>>
>>>>     doc/guides/rel_notes/release_19_02.rst        |  14 +
>>>>     doc/guides/sample_app_ug/ipsec_secgw.rst      | 159 +++++-
>>>>     examples/ipsec-secgw/Makefile                 |   5 +-
>>>>     examples/ipsec-secgw/ipsec-secgw.c            | 480 ++++++++++++++----
>>>>     examples/ipsec-secgw/ipsec.c                  | 101 ++--
>>>>     examples/ipsec-secgw/ipsec.h                  |  67 +++
>>>>     examples/ipsec-secgw/ipsec_process.c          | 357 +++++++++++++
>>>>     examples/ipsec-secgw/meson.build              |   6 +-
>>>>     examples/ipsec-secgw/parser.c                 |  91 ++++
>>>>     examples/ipsec-secgw/parser.h                 |   8 +-
>>>>     examples/ipsec-secgw/sa.c                     | 263 +++++++++-
>>>>     examples/ipsec-secgw/sp4.c                    |  35 +-
>>>>     examples/ipsec-secgw/sp6.c                    |  35 +-
>>>>     examples/ipsec-secgw/test/common_defs.sh      | 153 ++++++
>>>>     examples/ipsec-secgw/test/data_rxtx.sh        |  62 +++
>>>>     examples/ipsec-secgw/test/linux_test4.sh      |  63 +++
>>>>     examples/ipsec-secgw/test/linux_test6.sh      |  64 +++
>>>>     examples/ipsec-secgw/test/run_test.sh         |  80 +++
>>>>     .../test/trs_aescbc_sha1_common_defs.sh       |  69 +++
>>>>     .../ipsec-secgw/test/trs_aescbc_sha1_defs.sh  |  67 +++
>>>>     .../test/trs_aescbc_sha1_esn_atom_defs.sh     |   5 +
>>>>     .../test/trs_aescbc_sha1_esn_defs.sh          |  66 +++
>>>>     .../test/trs_aescbc_sha1_old_defs.sh          |   5 +
>>>>     .../test/trs_aesgcm_common_defs.sh            |  60 +++
>>>>     examples/ipsec-secgw/test/trs_aesgcm_defs.sh  |  66 +++
>>>>     .../test/trs_aesgcm_esn_atom_defs.sh          |   5 +
>>>>     .../ipsec-secgw/test/trs_aesgcm_esn_defs.sh   |  66 +++
>>>>     .../ipsec-secgw/test/trs_aesgcm_old_defs.sh   |   5 +
>>>>     .../test/tun_aescbc_sha1_common_defs.sh       |  68 +++
>>>>     .../ipsec-secgw/test/tun_aescbc_sha1_defs.sh  |  70 +++
>>>>     .../test/tun_aescbc_sha1_esn_atom_defs.sh     |   5 +
>>>>     .../test/tun_aescbc_sha1_esn_defs.sh          |  70 +++
>>>>     .../test/tun_aescbc_sha1_old_defs.sh          |   5 +
>>>>     .../test/tun_aesgcm_common_defs.sh            |  60 +++
>>>>     examples/ipsec-secgw/test/tun_aesgcm_defs.sh  |  70 +++
>>>>     .../test/tun_aesgcm_esn_atom_defs.sh          |   5 +
>>>>     .../ipsec-secgw/test/tun_aesgcm_esn_defs.sh   |  70 +++
>>>>     .../ipsec-secgw/test/tun_aesgcm_old_defs.sh   |   5 +
>>>>     38 files changed, 2727 insertions(+), 158 deletions(-)
>>>>     create mode 100644 examples/ipsec-secgw/ipsec_process.c
>>>>     create mode 100644 examples/ipsec-secgw/test/common_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/data_rxtx.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/linux_test4.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/linux_test6.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/run_test.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_common_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_esn_atom_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_esn_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_old_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_common_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_esn_atom_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_esn_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_old_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_common_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_esn_atom_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_esn_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_old_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_common_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_esn_atom_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_esn_defs.sh
>>>>     create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_old_defs.sh
>>>>