Re: [dpdk-dev] [PATCH v2] examples: add multi process crypto application

[Akhil Goyal <akhil.goyal@nxp.com>]

Hi Arek/Fiona,

> This patch adds example application that can test
> usage of cryptodev in multi process environment.
> More can be found in mp_crypto.rst in sample app guides.
> 
> Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
> ---

Shouldn't this be part of examples/multi_process?

>  doc/guides/sample_app_ug/index.rst     |    1 +
>  doc/guides/sample_app_ug/mp_crypto.rst |  153 +++++
>  examples/mp_crypto/Makefile            |   58 ++
>  examples/mp_crypto/main.c              | 1109
> ++++++++++++++++++++++++++++++++
>  examples/mp_crypto/meson.build         |   18 +
>  examples/mp_crypto/mp_crypto.c         |  139 ++++
>  examples/mp_crypto/mp_crypto.h         |  224 +++++++
>  examples/mp_crypto/mp_crypto_ipc.c     |   32 +
>  examples/mp_crypto/mp_crypto_parser.c  |  511 +++++++++++++++
>  examples/mp_crypto/mp_crypto_parser.h  |  149 +++++
>  examples/mp_crypto/mp_crypto_vectors.c |  175 +++++
>  examples/mp_crypto/mp_crypto_vectors.h |   66 ++
>  12 files changed, 2635 insertions(+)

Can we split this patch in some logical way so that it can be reviewed easily?

>  create mode 100644 doc/guides/sample_app_ug/mp_crypto.rst
>  create mode 100644 examples/mp_crypto/Makefile
>  create mode 100644 examples/mp_crypto/main.c
>  create mode 100644 examples/mp_crypto/meson.build
>  create mode 100644 examples/mp_crypto/mp_crypto.c
>  create mode 100644 examples/mp_crypto/mp_crypto.h
>  create mode 100644 examples/mp_crypto/mp_crypto_ipc.c
>  create mode 100644 examples/mp_crypto/mp_crypto_parser.c
>  create mode 100644 examples/mp_crypto/mp_crypto_parser.h
>  create mode 100644 examples/mp_crypto/mp_crypto_vectors.c
>  create mode 100644 examples/mp_crypto/mp_crypto_vectors.h
> 
> diff --git a/doc/guides/sample_app_ug/index.rst
> b/doc/guides/sample_app_ug/index.rst
> index affa9c5..ff033e4 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -35,6 +35,7 @@ Sample Applications User Guides
>      link_status_intr
>      server_node_efd
>      service_cores
> +    mp_crypto
>      multi_process
>      qos_metering
>      qos_scheduler
> diff --git a/doc/guides/sample_app_ug/mp_crypto.rst
> b/doc/guides/sample_app_ug/mp_crypto.rst
> new file mode 100644
> index 0000000..d3cb1d9
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/mp_crypto.rst
> @@ -0,0 +1,153 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2020 Intel Corporation.
> +
> +.. _mp_crypto:
> +
> +Multi-process Crypto Sample Application
> +=======================================
> +
> +The Multi-process Crypto application is a simple application that
> +allows to run crypto related operations in a multiple process environment. It
> +builds on the EAL primary/secondary process infrastructure.
> +
> +The application allows a user to configure devices, setup queue-pairs, create
> +and init sessions and specify data-path flow (enqueue/dequeue) in different
> +processes. The app can help to check if the PMD behaves correctly
> +in scenarios like the following:
> +
> +* device is configured in primary process, queue-pairs are setup in secondary
> process
> +
> +* queue pair is shared across processes, i.e. enqueue in one process and
> dequeue in another
> +
> +
Extra line

> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``mp_crypto`` sub-directory.
> +
> +Running the Application
> +-----------------------
> +
> +App binary: mp_crypto (in mp_crypto/build/app)
> +
> +For running PRIMARY or SECONDARY process standard EAL options apply:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto --proc-type primary
> +
> +    ./mp_crypto --proc-type secondary
> +
> +.. Note::
> +
> +	The same set of BDFs must be passed to all processes.
> +
> +.. Note::
> +	The same crypto devices must be created in all processes, e.g. in qat
> +	case if asym and sym devices are enabled in the primary process, they
> +	must be enabled in all secondary processes.
> +
> +General help can by checked by running:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- -h
> +
> +The application has a number of command line options:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- --devtype [dev-name]
> +
> +This option specifies which driver to use by its name (for example "crypto_qat").
> +The same name must be passed to all processes.
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- --config_dev [devA, devB,]
> +
> +This option specifies the list of devices that should be configured by this
> process,
> +this results in a call to the ``rte_cryptodev_configure`` API. devX is a positive
> +integer (including zero), the value is according to probe order (from the
> smallest
> +BDF number), not necessarily the cmdline order.
> +
> +Example command:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -w 03:01.2 -w 03:01.1 -w 03:01.3 --config-dev 0,2
> +
> +will configure devices 03:01.1 and 03:01.3.
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- --qp-config=[devA]:[qp_A, qp_B,];[devB]:[qp_A, qp_C];
> +
> +devX - positive integer (including zero), as in config_dev command
> +
> +qp_X - positive integer (including zero), specifies which queue pair should be
> setup
> +
> +This command specifies which queue pairs should be setup, resulting in a call to
> +``rte_cryptodev_queue_pair_setup`` API.
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -w 03:01.2 -w 03:01.1 -w 03:01.3 --qp-config="0:0,1;1:1;2:0,1;"
> +
> +This command will configure queue pairs 0 and 1 on device 0 (03:01.1), queue
> pair 1
> +on device 1 (03:01.2), queue pairs 0 and 1 on device 2 (03:01.3). The device in
> question
> +should be configured before that, though not necessarily by the same process.
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- --enq=[devX]:[qpX]:[ops]:[vector_id]
> +    ./mp_crypto -- --deq=[devX]:[qpX]:[ops]:[vector_id]
> +
> +devX - positive integer (including zero), as in config_dev command
> +
> +qp_X - positive integer (including zero), as in qp-config command
qpX

> +
> +ops - when positive integer - number of operations to enqueue/dequeue, when
> 0 infinite loop
> +
> +vector_id - positive integer (including zero), vector_id used by this process
What is vector_id? How is it meant to be used?

> +
> +This commands will enqueue/dequeue "ops" number of packets to qp_X on
> devX.
> +Example usage:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto -- --enq=2:0:0:0, --deq=2:0:0:0,
> +
> +Note. ',' comma character is necessary at the end due to some parser
> shortcomings.
> +
> +To close the application when running in an infinite loop a signal handler is
> +registered to catch interrupt signals i.e. ``ctrl-c`` should be used. When
> +used in primary process other processes will be notified about exiting
> +intention and will close after collecting remaining packets (if dequeuing).
> +
> +Example commands
> +----------------
> +
> +Use two different devices on 3 separate queues:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto --proc-type primary -c 1 -w 03:01.1 -w 03:01.2 -- --devtype
> "crypto_qat" --config-dev 0,1   --qp-config="0:0,1;1:0,1;" --session-mask=0x3  --
> enq=0:0:0:0, --deq=0:0:0:0,  --print-stats
> +    ./mp_crypto --proc-type secondary -c 2 -w 03:01.1 -w 03:01.2 -- --devtype
> "crypto_qat"  --enq=0:1:0:0, --deq=0:1:0:0,  --print-stats
> +    ./mp_crypto --proc-type secondary -c 4 -w 03:01.1 -w 03:01.2 -- --devtype
> "crypto_qat"  --enq=1:0:0:0, --deq=1:0:0:0,  --print-stats
> +
> +Use different processes to enqueue and dequeue to one queue pair:
> +
> +.. code-block:: console
> +
> +    ./mp_crypto --proc-type primary -c 1 -w 03:01.1 -- --devtype "crypto_qat" --
> config-dev 0    --session-mask=0x3 --qp-config="0:1;"   --enq=0:1:0:0,   --print-
> stats
> +    ./mp_crypto --proc-type secondary -c 2 -w 03:01.1 -- --devtype "crypto_qat"
> --deq=0:1:0:0,   --print-stats

What are the expected results out of this app. Can we have some logs in this doc?

Can we kill the primary app first? Will the secondary get killed automatically or will throw error?
Can we open and kill secondary apps multiple times without killing the primary?

This multi process app is only taking care of  crypto queues while others are for NICs.
Is it not worth to have crypto+NIC multi process app instead of this app?
I believe most common usecases of crypto are with network traffic.
Can we modify l2fwd-crypto for multi process?

> +
> +Limitations
> +-----------
> +
> +Software devices are not supported currently, but small changes in code suffice
> to enable it.
> +
> +Only one crypto vector and session type is possible to chose right now and it is
> AES-GCM test case.

Regards,
Akhil