[dpdk-dev] [PATCH v2 00/10] lib/librte_vhost: introduce new vhost user crypto backend support

[dpdk-dev] [PATCH v2 00/10] lib/librte_vhost: introduce new vhost user crypto backend support

[Fan Zhang]

This patchset adds crypto backend suppport to vhost_user library,
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

To make this patchset working, a few tweaks need to be done:

In the host:
1. Download the qemu source code, and apply the patches in:
http://lists.nongnu.org/archive/html/qemu-devel/2018-02/msg06448.html.

2. Recompile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12,13 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 3 lcores: 1 master, 1 worker,
and 1 statistics displaying worker. The application will create a UNIX socket
file /tmp/vhost_crypto1.socket.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0 

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/35383/) in the guest to work with
this patchset to achieve optimal performance. 

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.

Fan Zhang (10):
  lib/librte_vhost: add vhost user private info structure
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add head file
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update version map
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update prog guide and sample app guid

 doc/guides/prog_guide/vhost_lib.rst       |   27 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   83 ++
 examples/vhost_crypto/Makefile            |   59 ++
 examples/vhost_crypto/main.c              |  537 ++++++++++++
 lib/librte_vhost/Makefile                 |    9 +
 lib/librte_vhost/rte_vhost_crypto.h       |  122 +++
 lib/librte_vhost/rte_vhost_version.map    |   11 +
 lib/librte_vhost/vhost.h                  |    5 +-
 lib/librte_vhost/vhost_crypto.c           | 1280 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   15 +-
 lib/librte_vhost/vhost_user.h             |   38 +-
 12 files changed, 2184 insertions(+), 3 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.13.6

[dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Fan Zhang]

This patch adds a vhost_user_dev_priv structure and a vhost_user
message handler function prototype to vhost_user. This allows
different types of devices to add private information and their
device-specific vhost-user message function handlers to
virtio_net structure. The change to vhost_user_msg_handler is
also added to call the device-specific message handler.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost.h      |  5 ++++-
 lib/librte_vhost/vhost_user.c | 13 ++++++++++++-
 lib/librte_vhost/vhost_user.h |  7 +++++++
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index d947bc9e3..19ee3fd37 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -13,6 +13,7 @@
 #include <linux/virtio_net.h>
 #include <sys/socket.h>
 #include <linux/if.h>
+#include <stdbool.h>
 
 #include <rte_log.h>
 #include <rte_ether.h>
@@ -241,8 +242,10 @@ struct virtio_net {
 	struct guest_page       *guest_pages;
 
 	int			slave_req_fd;
-} __rte_cache_aligned;
 
+	/* Private data for different virtio device type */
+	void			*private_data;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 90ed2112e..6a90d2a96 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1477,7 +1477,18 @@ vhost_user_msg_handler(int vid, int fd)
 		break;
 
 	default:
-		ret = -1;
+		if (!dev->private_data)
+			ret = -1;
+		else {
+			struct vhost_user_dev_priv *priv = dev->private_data;
+
+			if (!priv->vhost_user_msg_handler)
+				ret = -1;
+			else {
+				ret = (*priv->vhost_user_msg_handler)(dev,
+						&msg, fd);
+			}
+		}
 		break;
 
 	}
diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index d4bd604b9..354615c8b 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -108,6 +108,13 @@ typedef struct VhostUserMsg {
 /* The version of the protocol we support */
 #define VHOST_USER_VERSION    0x1
 
+typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg,
+		int fd);
+
+struct vhost_user_dev_priv {
+	msg_handler vhost_user_msg_handler;
+	char data[0];
+};
 
 /* vhost_user.c */
 int vhost_user_msg_handler(int vid, int fd);
-- 
2.13.6

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Maxime Coquelin]

On 02/27/2018 05:29 PM, Fan Zhang wrote:
> This patch adds a vhost_user_dev_priv structure and a vhost_user
> message handler function prototype to vhost_user. This allows
> different types of devices to add private information and their
> device-specific vhost-user message function handlers to
> virtio_net structure. The change to vhost_user_msg_handler is
> also added to call the device-specific message handler.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>   lib/librte_vhost/vhost.h      |  5 ++++-
>   lib/librte_vhost/vhost_user.c | 13 ++++++++++++-
>   lib/librte_vhost/vhost_user.h |  7 +++++++
>   3 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
> index d947bc9e3..19ee3fd37 100644
> --- a/lib/librte_vhost/vhost.h
> +++ b/lib/librte_vhost/vhost.h
> @@ -13,6 +13,7 @@
>   #include <linux/virtio_net.h>
>   #include <sys/socket.h>
>   #include <linux/if.h>
> +#include <stdbool.h>

I think this include isn't needed looking at the rest of the patch.

>   
>   #include <rte_log.h>
>   #include <rte_ether.h>
> @@ -241,8 +242,10 @@ struct virtio_net {
>   	struct guest_page       *guest_pages;
>   
>   	int			slave_req_fd;
> -} __rte_cache_aligned;
>   
> +	/* Private data for different virtio device type */
> +	void			*private_data;
> +} __rte_cache_aligned;
>   
>   #define VHOST_LOG_PAGE	4096
>   
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 90ed2112e..6a90d2a96 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -1477,7 +1477,18 @@ vhost_user_msg_handler(int vid, int fd)
>   		break;
>   
>   	default:
> -		ret = -1;
> +		if (!dev->private_data)
> +			ret = -1;
> +		else {
> +			struct vhost_user_dev_priv *priv = dev->private_data;
> +
> +			if (!priv->vhost_user_msg_handler)
> +				ret = -1;
> +			else {
> +				ret = (*priv->vhost_user_msg_handler)(dev,
> +						&msg, fd);
> +			}
> +		}
>   		break;
>   
>   	}
> diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
> index d4bd604b9..354615c8b 100644
> --- a/lib/librte_vhost/vhost_user.h
> +++ b/lib/librte_vhost/vhost_user.h
> @@ -108,6 +108,13 @@ typedef struct VhostUserMsg {
>   /* The version of the protocol we support */
>   #define VHOST_USER_VERSION    0x1
>   
> +typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg,
> +		int fd);
> +
> +struct vhost_user_dev_priv {
> +	msg_handler vhost_user_msg_handler;
> +	char data[0];
> +};
>   
>   /* vhost_user.c */
>   int vhost_user_msg_handler(int vid, int fd);
> 

I think the wording is a bit misleading, I'm fine with having a
private_data pointer, but it should only be used by the external
backend.

Maybe what you need here is a new API to be to register a
callback for the external backend to handle specific requests.

Also, it might be interesting for the external backend to register
callbacks for existing requests. For example .pre_vhost_user_msg_handler
and .post_vhost_user_msg_handler. Doing so, the external backend could
for example catch beforehand any change that could affect resources
being used. Tomasz, Pawel, do you think that could help for the issue
you reported?

Thanks,
Maxime

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Zhang, Roy Fan]

Hi Maxime,

Thanks for the review.

> 
> I think this include isn't needed looking at the rest of the patch.

I agree. I will remove this line here.

> >   #define VHOST_USER_VERSION    0x1
> >
> > +typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg
> *msg,
> > +		int fd);
> > +
> > +struct vhost_user_dev_priv {
> > +	msg_handler vhost_user_msg_handler;
> > +	char data[0];
> > +};
> >
> >   /* vhost_user.c */
> >   int vhost_user_msg_handler(int vid, int fd);
> >
> 
> I think the wording is a bit misleading, I'm fine with having a private_data
> pointer, but it should only be used by the external backend.
> 
> Maybe what you need here is a new API to be to register a callback for the
> external backend to handle specific requests.

That's exactly what I need. 
Shall I rework the code like this?

/* vhost.h */
struct virtio_net {
	....
	void *extern_data; /*<< private data for external backend */
	
}

/* vhost_user.h */
typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg,
		int fd);

struct vhost_user_dev_extern {
	msg_handler post_vhost_user_msg_handler;
	char data[0];
};

int 
vhost_user_register_call_back(struct virtio_net *dev, msg_handler post_msg_handler);

> 
> Also, it might be interesting for the external backend to register callbacks for
> existing requests. For example .pre_vhost_user_msg_handler
> and .post_vhost_user_msg_handler. Doing so, the external backend could
> for example catch beforehand any change that could affect resources being
> used. Tomasz, Pawel, do you think that could help for the issue you reported?
> 


I think it is definitely a good idea. However there will be a problem. As vhost_crypto does not require pre_vhost_user_msg_handler I think it may not appropriate to add pre_vhost_user_msg_handler in this patchset. 

Thanks a million Maxime.

> Thanks,
> Maxime

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Wodkowski, PawelX]

> int
> vhost_user_register_call_back(struct virtio_net *dev, msg_handler
> post_msg_handler);
> 
> >
> > Also, it might be interesting for the external backend to register callbacks for
> > existing requests. For example .pre_vhost_user_msg_handler
> > and .post_vhost_user_msg_handler. Doing so, the external backend could
> > for example catch beforehand any change that could affect resources being
> > used. Tomasz, Pawel, do you think that could help for the issue you reported?
> >
> 

Sorry, I didn't see this patch series somehow. I need to fix my mail filters.
This is definitely good idea to introduce pre and vhost user post message handlers.

> 
> I think it is definitely a good idea. However there will be a problem. As
> vhost_crypto does not require pre_vhost_user_msg_handler I think it may not
> appropriate to add pre_vhost_user_msg_handler in this patchset.
> 
> Thanks a million Maxime.
> 
> > Thanks,
> > Maxime

Paweł

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Maxime Coquelin]

Hi,

On 03/21/2018 10:10 AM, Zhang, Roy Fan wrote:
> Hi Maxime,
> 
> Thanks for the review.
> 
>>
>> I think this include isn't needed looking at the rest of the patch.
> 
> I agree. I will remove this line here.
> 
>>>    #define VHOST_USER_VERSION    0x1
>>>
>>> +typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg
>> *msg,
>>> +		int fd);
>>> +
>>> +struct vhost_user_dev_priv {
>>> +	msg_handler vhost_user_msg_handler;
>>> +	char data[0];
>>> +};
>>>
>>>    /* vhost_user.c */
>>>    int vhost_user_msg_handler(int vid, int fd);
>>>
>>
>> I think the wording is a bit misleading, I'm fine with having a private_data
>> pointer, but it should only be used by the external backend.
>>
>> Maybe what you need here is a new API to be to register a callback for the
>> external backend to handle specific requests.
> 
> That's exactly what I need.
> Shall I rework the code like this?

These new API are to be placed in rte_vhost.h, else it won't be
exported.

> /* vhost.h */
> struct virtio_net {
> 	....
> 	void *extern_data; /*<< private data for external backend */
> 	
> }

Looks good, you may need to add getter and setter APIs as struct
virtio_net isn't part of the API.

> /* vhost_user.h */
> typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg,
> 		int fd);

I wonder if the fd is really necessary, as if a reply is to be sent, it
can be done by the vhost lib.

> struct vhost_user_dev_extern {
> 	msg_handler post_vhost_user_msg_handler;
> 	char data[0];
Why is this filed needed?
> };
I would change this to:
struct rte_vhost_user_dev_extern_ops {
	rte_vhost_msg_handler pre_vhost_user_msg_handler;
	rte_vhost_msg_handler post_vhost_user_msg_handler;
};

> int
> vhost_user_register_call_back(struct virtio_net *dev, msg_handler post_msg_handler);

and something like:
rte_vhost_user_register_extern_ops(struct virtio_net *dev, struct
				rte_vhost_user_dev_extern_ops *ops);

>>
>> Also, it might be interesting for the external backend to register callbacks for
>> existing requests. For example .pre_vhost_user_msg_handler
>> and .post_vhost_user_msg_handler. Doing so, the external backend could
>> for example catch beforehand any change that could affect resources being
>> used. Tomasz, Pawel, do you think that could help for the issue you reported?
>>
> 
> 
> I think it is definitely a good idea. However there will be a problem. As vhost_crypto does not require pre_vhost_user_msg_handler I think it may not appropriate to add pre_vhost_user_msg_handler in this patchset.

I see, but please add the pre_ callback directly in this series, as we
know it will be useful (thanks Pawel).
It will avoid breaking the API again when we'll need it.

Thanks,
Maxime
> Thanks a million Maxime.
> 
>> Thanks,
>> Maxime

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Zhang, Roy Fan]

Hi Maxime,

Thanks a lot for the fast reply. 

> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Wednesday, March 21, 2018 1:03 PM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org; Kulasek,
> TomaszX <tomaszx.kulasek@intel.com>; Wodkowski, PawelX
> <pawelx.wodkowski@intel.com>
> Cc: jianjay.zhou@huawei.com; yliu@fridaylinux.org; Tan, Jianfeng
> <jianfeng.tan@intel.com>; Bie, Tiwei <tiwei.bie@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user
> private info structure
> 
> Hi,
> 
> On 03/21/2018 10:10 AM, Zhang, Roy Fan wrote:
> > Hi Maxime,
> >
> > Thanks for the review.
> >
> >>
> >> I think this include isn't needed looking at the rest of the patch.
> >
> > I agree. I will remove this line here.
> >
> >>>    #define VHOST_USER_VERSION    0x1
> >>>
> >>> +typedef int (*msg_handler)(struct virtio_net *dev, struct
> >>> +VhostUserMsg
> >> *msg,
> >>> +		int fd);
> >>> +
> >>> +struct vhost_user_dev_priv {
> >>> +	msg_handler vhost_user_msg_handler;
> >>> +	char data[0];
> >>> +};
> >>>
> >>>    /* vhost_user.c */
> >>>    int vhost_user_msg_handler(int vid, int fd);
> >>>
> >>
> >> I think the wording is a bit misleading, I'm fine with having a
> >> private_data pointer, but it should only be used by the external backend.
> >>
> >> Maybe what you need here is a new API to be to register a callback
> >> for the external backend to handle specific requests.
> >
> > That's exactly what I need.
> > Shall I rework the code like this?
> 
> These new API are to be placed in rte_vhost.h, else it won't be exported.
> 
> > /* vhost.h */
> > struct virtio_net {
> > 	....
> > 	void *extern_data; /*<< private data for external backend */
> >
> > }
> 
> Looks good, you may need to add getter and setter APIs as struct virtio_net
> isn't part of the API.
> 
> > /* vhost_user.h */
> > typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg
> *msg,
> > 		int fd);
> 
> I wonder if the fd is really necessary, as if a reply is to be sent, it can be done
> by the vhost lib.

Same as the messages VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_GET_VRING_BASE etc, the external backend such as vhost crypto will require to send an reply independently. 
In case of vhost crypto, the message create_session will require the backend to send a session id back to the frontend. 

I tried to set VHOST_USER_NEED_REPLY bit in the vhost crypto message handler but it will cause problem, qemu returns " qemu-system-x86_64: Received bad msg size."

Another solution is to pass a variable back from the message handler to indicate sending the reply instead.
So the function prototype can be 
typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg, uint32_t *require_reply);


> > struct vhost_user_dev_extern {
> > 	msg_handler post_vhost_user_msg_handler;
> > 	char data[0];
> Why is this filed needed?

Either this way, or using a pointer. The idea is to allocate contiguous memory when creating new vhost_user_dev_extern structure. The former way is slightly more performant to avoid another memory read, but I can live with the pointer instead :-)

> > };
> I would change this to:
> struct rte_vhost_user_dev_extern_ops {
> 	rte_vhost_msg_handler pre_vhost_user_msg_handler;
> 	rte_vhost_msg_handler post_vhost_user_msg_handler; };
> 
> > int
> > vhost_user_register_call_back(struct virtio_net *dev, msg_handler
> > post_msg_handler);
> 
> and something like:
> rte_vhost_user_register_extern_ops(struct virtio_net *dev, struct
> 				rte_vhost_user_dev_extern_ops *ops);

Great idea!

> >>
> >> Also, it might be interesting for the external backend to register
> >> callbacks for existing requests. For example
> >> .pre_vhost_user_msg_handler and .post_vhost_user_msg_handler.
> Doing
> >> so, the external backend could for example catch beforehand any
> >> change that could affect resources being used. Tomasz, Pawel, do you
> think that could help for the issue you reported?
> >>
> >
> >
> > I think it is definitely a good idea. However there will be a problem. As
> vhost_crypto does not require pre_vhost_user_msg_handler I think it may
> not appropriate to add pre_vhost_user_msg_handler in this patchset.
> 
> I see, but please add the pre_ callback directly in this series, as we know it
> will be useful (thanks Pawel).
> It will avoid breaking the API again when we'll need it.

Will do.

> 
> Thanks,
> Maxime
> > Thanks a million Maxime.
> >
> >> Thanks,
> >> Maxime

Thanks,
Fan

Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user private info structure

[Wodkowski, PawelX]

> -----Original Message-----
> From: Zhang, Roy Fan
> Sent: Wednesday, March 21, 2018 5:12 PM
> To: Maxime Coquelin <maxime.coquelin@redhat.com>; dev@dpdk.org;
> Kulasek, TomaszX <tomaszx.kulasek@intel.com>; Wodkowski, PawelX
> <pawelx.wodkowski@intel.com>
> Cc: jianjay.zhou@huawei.com; yliu@fridaylinux.org; Tan, Jianfeng
> <jianfeng.tan@intel.com>; Bie, Tiwei <tiwei.bie@intel.com>
> Subject: RE: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user
> private info structure
> 
> Hi Maxime,
> 
> Thanks a lot for the fast reply.
> 
> > -----Original Message-----
> > From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> > Sent: Wednesday, March 21, 2018 1:03 PM
> > To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org; Kulasek,
> > TomaszX <tomaszx.kulasek@intel.com>; Wodkowski, PawelX
> > <pawelx.wodkowski@intel.com>
> > Cc: jianjay.zhou@huawei.com; yliu@fridaylinux.org; Tan, Jianfeng
> > <jianfeng.tan@intel.com>; Bie, Tiwei <tiwei.bie@intel.com>
> > Subject: Re: [dpdk-dev] [PATCH v2 01/10] lib/librte_vhost: add vhost user
> > private info structure
> >
> > Hi,
> >
> > On 03/21/2018 10:10 AM, Zhang, Roy Fan wrote:
> > > Hi Maxime,
> > >
> > > Thanks for the review.
> > >
> > >>
> > >> I think this include isn't needed looking at the rest of the patch.
> > >
> > > I agree. I will remove this line here.
> > >
> > >>>    #define VHOST_USER_VERSION    0x1
> > >>>
> > >>> +typedef int (*msg_handler)(struct virtio_net *dev, struct
> > >>> +VhostUserMsg
> > >> *msg,
> > >>> +		int fd);
> > >>> +
> > >>> +struct vhost_user_dev_priv {
> > >>> +	msg_handler vhost_user_msg_handler;
> > >>> +	char data[0];
> > >>> +};
> > >>>
> > >>>    /* vhost_user.c */
> > >>>    int vhost_user_msg_handler(int vid, int fd);
> > >>>
> > >>
> > >> I think the wording is a bit misleading, I'm fine with having a
> > >> private_data pointer, but it should only be used by the external backend.
> > >>
> > >> Maybe what you need here is a new API to be to register a callback
> > >> for the external backend to handle specific requests.
> > >
> > > That's exactly what I need.
> > > Shall I rework the code like this?
> >
> > These new API are to be placed in rte_vhost.h, else it won't be exported.
> >
> > > /* vhost.h */
> > > struct virtio_net {
> > > 	....
> > > 	void *extern_data; /*<< private data for external backend */
> > >
> > > }
> >
> > Looks good, you may need to add getter and setter APIs as struct virtio_net
> > isn't part of the API.
> >
> > > /* vhost_user.h */
> > > typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg
> > *msg,
> > > 		int fd);
> >
> > I wonder if the fd is really necessary, as if a reply is to be sent, it can be done
> > by the vhost lib.
> 
> Same as the messages VHOST_USER_GET_PROTOCOL_FEATURES and
> VHOST_USER_GET_VRING_BASE etc, the external backend such as vhost crypto
> will require to send an reply independently.
> In case of vhost crypto, the message create_session will require the backend to
> send a session id back to the frontend.
> 
> I tried to set VHOST_USER_NEED_REPLY bit in the vhost crypto message
> handler but it will cause problem, qemu returns " qemu-system-x86_64:
> Received bad msg size."
> 
> Another solution is to pass a variable back from the message handler to
> indicate sending the reply instead.
> So the function prototype can be
> typedef int (*msg_handler)(struct virtio_net *dev, struct VhostUserMsg *msg,
> uint32_t *require_reply);
> 
> 
> > > struct vhost_user_dev_extern {
> > > 	msg_handler post_vhost_user_msg_handler;
> > > 	char data[0];
> > Why is this filed needed?
> 
> Either this way, or using a pointer. The idea is to allocate contiguous memory
> when creating new vhost_user_dev_extern structure. The former way is slightly
> more performant to avoid another memory read, but I can live with the pointer
> instead :-)
> 
> > > };
> > I would change this to:
> > struct rte_vhost_user_dev_extern_ops {
> > 	rte_vhost_msg_handler pre_vhost_user_msg_handler;
> > 	rte_vhost_msg_handler post_vhost_user_msg_handler; };
> >
> > > int
> > > vhost_user_register_call_back(struct virtio_net *dev, msg_handler
> > > post_msg_handler);
> >
> > and something like:
> > rte_vhost_user_register_extern_ops(struct virtio_net *dev, struct
> > 				rte_vhost_user_dev_extern_ops *ops);
> 
> Great idea!

Can we add this handler in struct vhost_device_ops? There is filed void 
*reserved[2]. We can use first element to define this handler.

> 
> > >>
> > >> Also, it might be interesting for the external backend to register
> > >> callbacks for existing requests. For example
> > >> .pre_vhost_user_msg_handler and .post_vhost_user_msg_handler.
> > Doing
> > >> so, the external backend could for example catch beforehand any
> > >> change that could affect resources being used. Tomasz, Pawel, do you
> > think that could help for the issue you reported?
> > >>
> > >
> > >
> > > I think it is definitely a good idea. However there will be a problem. As
> > vhost_crypto does not require pre_vhost_user_msg_handler I think it may
> > not appropriate to add pre_vhost_user_msg_handler in this patchset.
> >
> > I see, but please add the pre_ callback directly in this series, as we know it
> > will be useful (thanks Pawel).
> > It will avoid breaking the API again when we'll need it.
> 
> Will do.
> 
> >
> > Thanks,
> > Maxime
> > > Thanks a million Maxime.
> > >
> > >> Thanks,
> > >> Maxime
> 
> Thanks,
> Fan

[dpdk-dev] [PATCH v2 02/10] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_user.c |  2 ++
 lib/librte_vhost/vhost_user.h | 31 ++++++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 6a90d2a96..9d736a24f 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -50,6 +50,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index 354615c8b..99febb7fa 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -20,13 +20,15 @@
 #define VHOST_USER_PROTOCOL_F_REPLY_ACK	3
 #define VHOST_USER_PROTOCOL_F_NET_MTU 4
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
 
 #define VHOST_USER_PROTOCOL_FEATURES	((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -52,6 +54,8 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
 	VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -79,6 +83,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		VhostUserRequest master;
@@ -99,6 +127,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
-- 
2.13.6

[dpdk-dev] [PATCH v2 03/10] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 399 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 399 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 000000000..b7b7ff39d
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,399 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <rte_malloc.h>
+#include <rte_jhash.h>
+#include <rte_mbuf.h>
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#include <rte_hexdump.h>
+#endif
+#include "vhost.h"
+#include "vhost_user.h"
+#include "rte_vhost_crypto.h"
+
+#define NB_MEMPOOL_OBJS			(1024)
+#define NB_CRYPTO_DESCRIPTORS		(1024)
+#define NB_CACHE_OBJS			(128)
+
+#define SESSION_MAP_ENTRIES		(1024) /**< Max nb sessions per vdev */
+#define MAX_KEY_SIZE			(32)
+#define VHOST_CRYPTO_MAX_IV_LEN		(16)
+#define MAX_COUNT_DOWN_TIMES		(100)
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_HASH) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
+/**
+ * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and VIRTIO_CRYPTO_*ALGO*, for
+ * algorithms not supported by RTE_CRYPTODEV, the -VIRTIO_CRYPTO_NOTSUPP is
+ * returned.
+ */
+static int cipher_algo_transform[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_ARC4,
+		RTE_CRYPTO_CIPHER_AES_ECB,
+		RTE_CRYPTO_CIPHER_AES_CBC,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_CIPHER_DES_ECB */
+		RTE_CRYPTO_CIPHER_DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_ECB,
+		RTE_CRYPTO_CIPHER_3DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_CTR,
+		RTE_CRYPTO_CIPHER_KASUMI_F8,
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
+		RTE_CRYPTO_CIPHER_AES_F8,
+		RTE_CRYPTO_CIPHER_AES_XTS,
+		RTE_CRYPTO_CIPHER_ZUC_EEA3
+};
+
+/**
+ * VIRTIO_CRYTPO_AUTH_* indexes are not sequential, the gaps are filled with
+ * -VIRTIO_CRYPTO_BADMSG errors.
+ */
+static int auth_algo_transform[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_MD5_HMAC,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_SHA224_HMAC,
+		RTE_CRYPTO_AUTH_SHA256_HMAC,
+		RTE_CRYPTO_AUTH_SHA384_HMAC,
+		RTE_CRYPTO_AUTH_SHA512_HMAC,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CMAC_3DES */
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_KASUMI_F9,
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		RTE_CRYPTO_AUTH_AES_GMAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_GMAC_TWOFISH */
+		RTE_CRYPTO_AUTH_AES_CBC_MAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 */
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC
+};
+
+static int cipher_op_transform[] = {
+		-VIRTIO_CRYPTO_BADMSG, /* meaningless */
+		RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+		RTE_CRYPTO_CIPHER_OP_DECRYPT
+};
+
+static int iv_lens[] = {
+		-1, /* Invalid input */
+		0, /* RTE_CRYPTO_CIPHER_NULL */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CBC */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CTR */
+		8, /* RTE_CRYPTO_CIPHER_3DES_ECB */
+		16, /* RTE_CRYPTO_CIPHER_AES_CBC */
+		/* TODO: add common algos */
+};
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t zero_copy;
+} __rte_cache_aligned;
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	ret = cipher_op_transform[param->dir];
+	if (unlikely(ret < 0)) {
+		VC_LOG_DBG("Bad operation type");
+		return ret;
+	}
+	xform->cipher.op = ret;
+	ret = iv_lens[xform->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = iv_lens[xform_cipher->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform[param->hash_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_handler(struct virtio_net *dev, struct VhostUserMsg *msg,
+		int fd)
+{
+	struct vhost_user_dev_priv *priv;
+	struct vhost_crypto *vcrypto;
+	int ret = 0;
+
+	priv = dev->private_data;
+	if (unlikely(priv == NULL)) {
+		VC_LOG_ERR("Cannot find Vhost-Crypto private data");
+		return -ENOENT;
+	}
+	vcrypto = (struct vhost_crypto *)priv->data;
+
+	if (msg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto, &msg->payload.crypto_session);
+
+		msg->flags &= ~VHOST_USER_VERSION_MASK;
+		msg->flags &= ~VHOST_USER_NEED_REPLY;
+		msg->flags |= VHOST_USER_VERSION;
+		msg->flags |= VHOST_USER_REPLY_MASK;
+
+		ret = send_fd_message(fd, (char *)msg, VHOST_USER_HDR_SIZE +
+				msg->size, NULL, 0);
+	} else if (msg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, msg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
-- 
2.13.6

[dpdk-dev] [PATCH v2 04/10] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 607 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 607 insertions(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index b7b7ff39d..3b6323cda 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -21,6 +21,10 @@
 #define VHOST_CRYPTO_MAX_IV_LEN		(16)
 #define MAX_COUNT_DOWN_TIMES		(100)
 
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
 #ifdef RTE_LIBRTE_VHOST_DEBUG
 #define VC_LOG_ERR(fmt, args...)				\
 	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
@@ -49,6 +53,12 @@
 		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |			\
 		(1 << VIRTIO_NET_F_CTRL_VQ))
 
+
+#define GPA_TO_VVA(t, m, a)	(t)(uintptr_t)rte_vhost_gpa_to_vva(m, a)
+
+/* Macro to get the buffer at the end of rte_crypto_op */
+#define REQ_OP_OFFSET		(IV_OFFSET + VHOST_CRYPTO_MAX_IV_LEN)
+
 /**
  * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and VIRTIO_CRYPTO_*ALGO*, for
  * algorithms not supported by RTE_CRYPTODEV, the -VIRTIO_CRYPTO_NOTSUPP is
@@ -170,6 +180,23 @@ struct vhost_crypto {
 	uint8_t zero_copy;
 } __rte_cache_aligned;
 
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+
+	uint16_t desc_idx;
+	uint32_t len;
+	struct vhost_virtqueue *vq;
+
+	uint8_t zero_copy;
+
+	int vid;
+
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+};
+
 static int
 transform_cipher_param(struct rte_crypto_sym_xform *xform,
 		VhostUserCryptoSessionParam *param)
@@ -397,3 +424,583 @@ vhost_crypto_msg_handler(struct virtio_net *dev, struct VhostUserMsg *msg,
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	if (vcrypto->zero_copy != 0) {
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+	} else {
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	if (vcrypto->zero_copy != 0) {
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+	} else {
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	if (vcrypto->zero_copy != 0) {
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+	} else {
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	if (vcrypto->zero_copy != 0) {
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+	} else {
+		uint32_t digest_offset = m_dst->data_len;
+		void *digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->zero_copy;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.13.6

[dpdk-dev] [PATCH v2 05/10] lib/librte_vhost: add head file

[Fan Zhang]

This patch adds public head file API for vhost crypto

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h | 122 ++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 000000000..1560fcc12
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+#include <linux/virtio_crypto.h>
+#include <rte_cryptodev.h>
+#include <rte_hash.h>
+#include <rte_pause.h>
+#include "rte_vhost.h"
+
+#ifndef MAX_DATA_QUEUES
+#define MAX_DATA_QUEUES	(1)
+#endif
+
+#define VIRTIO_CRYPTO_CTRL_QUEUE	(0)
+#define VIRTIO_CRYPTO_MAX_NUM_DEVS	(64)
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS	(64)
+
+/** Feature bits */
+#define VIRTIO_CRYPTO_F_CIPHER_SESSION_MODE	(1)
+#define VIRTIO_CRYPTO_F_HASH_SESSION_MODE	(2)
+#define VIRTIO_CRYPTO_F_MAC_SESSION_MODE	(3)
+#define VIRTIO_CRYPTO_F_AEAD_SESSION_MODE	(4)
+
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param enable_zc
+ *  Flag of zero copy feature, set 1 to enable or 0 to disable.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_set_zero_copy(int vid, uint32_t enable_zc);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and convert to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the converted crypto ops are
+ * dequeued from the cryptodev, this function shall be called to update the
+ * used-ring indexes and write the processed data back to the vring descriptor
+ * (if zero-copy is disabled).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The pointer to the array that the owner callfd number(s) of the
+ *  virtio-crypto requests contained in the cryptodev operations finalized to
+ *  be written back. The size of the array shall be no less than the number
+ *  of total virtual devices possible.
+ * @nb_callfds
+ *  The number of call_fd numbers contained in the callfds array.
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
+#endif /**< _VHOST_CRYPTO_H_ */
-- 
2.13.6

[dpdk-dev] [PATCH v2 06/10] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 274 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 274 insertions(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 3b6323cda..3233953be 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -1004,3 +1004,277 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_hash_parameters params = {0};
+	struct vhost_user_dev_priv *priv;
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (vid >= VIRTIO_CRYPTO_MAX_NUM_DEVS || !dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	priv = rte_zmalloc_socket(NULL, sizeof(*priv) + sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!priv) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto = (struct vhost_crypto *)priv->data;
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	/** by default the zero copy is disabled, thus we need big enough
+	 *  mbufs.
+	 */
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_BUF_SIZE * 2, rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	priv->vhost_user_msg_handler = vhost_crypto_msg_handler;
+	dev->private_data = (void *)priv;
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(priv);
+
+	return ret;
+}
+
+int
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_user_dev_priv *priv;
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	priv = dev->private_data;
+	if (unlikely(priv == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vcrypto = (struct vhost_crypto *)priv->data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(priv);
+
+	dev->private_data = NULL;
+
+	return 0;
+}
+
+int
+rte_vhost_crypto_set_zero_copy(int vid, uint32_t enable_zc)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_user_dev_priv *priv;
+	struct vhost_crypto *vcrypto;
+	uint32_t buf_size;
+	char name[128];
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	priv = dev->private_data;
+	if (unlikely(priv == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vcrypto = (struct vhost_crypto *)priv->data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->zero_copy == (uint8_t)enable_zc)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	buf_size = enable_zc ? 0 : (RTE_MBUF_DEFAULT_BUF_SIZE * 2);
+
+	rte_mempool_free(vcrypto->mbuf_pool);
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req), buf_size,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		return -ENOMEM;
+	}
+
+	vcrypto->zero_copy = enable_zc ? 1 : 0;
+
+	return 0;
+}
+
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_user_dev_priv *priv;
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	priv = dev->private_data;
+	if (unlikely(priv == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vcrypto = (struct vhost_crypto *)priv->data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.13.6

[dpdk-dev] [PATCH v2 07/10] lib/librte_vhost: update version map

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_version.map | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index df0103129..e54e9d64a 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -59,3 +59,14 @@ DPDK_18.02 {
 	rte_vhost_vring_call;
 
 } DPDK_17.08;
+
+DPDK_18.05 {
+	global:
+
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_set_zero_copy;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+
+} DPDK_18.02;
-- 
2.13.6

[dpdk-dev] [PATCH v2 08/10] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 5d6c6abae..f0dd3f3c3 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -19,12 +19,21 @@ ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
 LDLIBS += -lnuma
 endif
 LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+LDLIBS += -lrte_cryptodev
+endif
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
 					vhost_user.c virtio_net.c
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto.c
+endif
 
 # install includes
 SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost_crypto.h
+endif
 
 include $(RTE_SDK)/mk/rte.lib.mk
-- 
2.13.6

[dpdk-dev] [PATCH v2 09/10] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile |  59 +++++
 examples/vhost_crypto/main.c   | 537 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 596 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 000000000..e45752453
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,59 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 000000000..3dae84948
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,537 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer	 in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Intel Corporation nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy = 1;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
-- 
2.13.6

[dpdk-dev] [PATCH v2 10/10] doc: update prog guide and sample app guid

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 27 ++++++++++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 83 +++++++++++++++++++++++++++++++
 3 files changed, 111 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6a3..f03b9aad8 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,33 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of `new_device()`, this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of `cryptodev_id`.
+
+* ``rte_vhost_crypto_free(vid)``
+
+  As an extension of `destroy_device()`, this function frees up the memory
+  created in `rte_vhost_crypto_create()`.
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) `nb_ops` virtio-crypto requests from the virt-queue
+  `queue_id` of the guest `vid`, converts them to DPDK Crypto Operations, and
+  fills the `ops` with conversion results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops, callfds, nb_fds)``
+
+  After the `ops` are dequeued from Cryptodev, updates the used ring
+  of the virtqueue, copy back the data (if zero-copy is disabled), and updates
+  the callfd numbers for the application to notify the guest(s).
+
+* ``rte_vhost_crypto_set_zero_copy(vid, enable_zc)``
+  This function enables/disables the zero copy feature of the vhost crypto
+  acceleration engine.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda80..57e8354d4 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 000000000..5051064dc
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,83 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2010-2014 Intel Corporation.
+
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
-- 
2.13.6