[dpdk-dev] DPDK Accelaration Enhancement

[dpdk-dev] DPDK Accelaration Enhancement

[Ant loves honey]

Currently DPDK has the crypto PMD (.../drivers/crypto/qat/) and example code (.../examples/dpdk_qat).
Intel QuickAssist Technology also supports compression along with crypto.  Last weekend, there is a proposed project to Intel Dev Mesh - "A VPP plugin utilizing Intel QucikAssist Technology to perform hardware assisted compression operations"

https://devmesh.intel.com/projects/a-vpp-plugin-utilizing-intel-quickassist-technology-to-perform-hardware-assisted-compression-operation

Do we need a new PMD driver for compression or should we modify the exiting crypto/qat driver to also support compression?  

The Intel QAT driver should be present in any Linux kernel greater than version 4.4

I am trying to put the pieces together and hitting a roadblock. I am also figuring how the PMD driver is interacting with the Intel QAT driver at the code level.
Any pointer on how to move forward is greatly appreciated.
Please also let me know if I should modify the proposed project.

Merry Christmas and Happy New Year,
Anthony.

Re: [dpdk-dev] DPDK Accelaration Enhancement

[Jain, Deepak K]

HI Anthony,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ant loves honey
> Sent: Saturday, December 24, 2016 10:56 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] DPDK Accelaration Enhancement
> 
> Currently DPDK has the crypto PMD (.../drivers/crypto/qat/) and example
> code (.../examples/dpdk_qat).
> Intel QuickAssist Technology also supports compression along with
> crypto.  Last weekend, there is a proposed project to Intel Dev Mesh - "A
> VPP plugin utilizing Intel QucikAssist Technology to perform hardware
> assisted compression operations"
> 
> https://devmesh.intel.com/projects/a-vpp-plugin-utilizing-intel-quickassist-
> technology-to-perform-hardware-assisted-compression-operation
> 
> Do we need a new PMD driver for compression or should we modify the
> exiting crypto/qat driver to also support compression?

There is still NO compression support in DPDK which utilizes Intel(R) QuickAssist Technology.

> 
> The Intel QAT driver should be present in any Linux kernel greater than
> version 4.4
> 
> I am trying to put the pieces together and hitting a roadblock. I am also
> figuring how the PMD driver is interacting with the Intel QAT driver at the
> code level.
> Any pointer on how to move forward is greatly appreciated.
> Please also let me know if I should modify the proposed project.
> 

Have you gone through the Crypto Documentation on dpdk.org and release notes (qat.rst) for more information?

> Merry Christmas and Happy New Year,
> Anthony.

Regards,
Deepak

Re: [dpdk-dev] DPDK Accelaration Enhancement

[Ant loves honey]

Deepak,
The .../doc/guides/cryptodevs/qat.rst is geared toward crypto and no mention of compression.
I am trying to figure out what it takes to have compression support on DPDK such as new PMD driver, additional defines and/or API or chipset initialization since the Intel QAT can support compression.
It seems to me IP payload can make good use of the hardware assisted compression and this will be a nice feature to have.
Thanks so much,
anthony.

      From: "Jain, Deepak K" <deepak.k.jain@intel.com>
 To: Ant loves honey <ant_love_honey@yahoo.com>; "dev@dpdk.org" <dev@dpdk.org> 
 Sent: Sunday, December 25, 2016 10:02 PM
 Subject: RE: [dpdk-dev] DPDK Accelaration Enhancement
   
HI Anthony,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ant loves honey
> Sent: Saturday, December 24, 2016 10:56 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] DPDK Accelaration Enhancement
> 
> Currently DPDK has the crypto PMD (.../drivers/crypto/qat/) and example
> code (.../examples/dpdk_qat).
> Intel QuickAssist Technology also supports compression along with
> crypto.  Last weekend, there is a proposed project to Intel Dev Mesh - "A
> VPP plugin utilizing Intel QucikAssist Technology to perform hardware
> assisted compression operations"
> 
> https://devmesh.intel.com/projects/a-vpp-plugin-utilizing-intel-quickassist-
> technology-to-perform-hardware-assisted-compression-operation
> 
> Do we need a new PMD driver for compression or should we modify the
> exiting crypto/qat driver to also support compression?

There is still NO compression support in DPDK which utilizes Intel(R) QuickAssist Technology.

> 
> The Intel QAT driver should be present in any Linux kernel greater than
> version 4.4
> 
> I am trying to put the pieces together and hitting a roadblock. I am also
> figuring how the PMD driver is interacting with the Intel QAT driver at the
> code level.
> Any pointer on how to move forward is greatly appreciated.
> Please also let me know if I should modify the proposed project.
> 

Have you gone through the Crypto Documentation on dpdk.org and release notes (qat.rst) for more information?

> Merry Christmas and Happy New Year,
> Anthony.

Regards,
Deepak


   
From yuanhan.liu@linux.intel.com  Mon Dec 26 07:25:34 2016
Return-Path: <yuanhan.liu@linux.intel.com>
Received: from mga02.intel.com (mga02.intel.com [134.134.136.20])
 by dpdk.org (Postfix) with ESMTP id 4FFC398
 for <dev@dpdk.org>; Mon, 26 Dec 2016 07:25:34 +0100 (CET)
Received: from fmsmga001.fm.intel.com ([10.253.24.23])
 by orsmga101.jf.intel.com with ESMTP; 25 Dec 2016 22:25:33 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.33,404,1477983600"; d="scan'208";a="1086663544"
Received: from yliu-dev.sh.intel.com (HELO yliu-dev) ([10.239.67.162])
 by fmsmga001.fm.intel.com with ESMTP; 25 Dec 2016 22:25:32 -0800
Date: Mon, 26 Dec 2016 14:27:19 +0800
From: Yuanhan Liu <yuanhan.liu@linux.intel.com>
To: Jianfeng Tan <jianfeng.tan@intel.com>
Cc: dev@dpdk.org, ferruh.yigit@intel.com, cunming.liang@intel.com
Message-ID: <20161226062719.GA19288@yliu-dev.sh.intel.com>
References: <1480689075-66977-1-git-send-email-jianfeng.tan@intel.com>
 <1482477266-39199-1-git-send-email-jianfeng.tan@intel.com>
 <1482477266-39199-3-git-send-email-jianfeng.tan@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1482477266-39199-3-git-send-email-jianfeng.tan@intel.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Subject: Re: [dpdk-dev] [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK
	notification
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Dec 2016 06:25:34 -0000

On Fri, Dec 23, 2016 at 07:14:21AM +0000, Jianfeng Tan wrote:
> In driver probe phase, we obtain device information; and then virtio
> driver will initialize device and stores info, like negotiated
> features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
> notification from virtio driver, and then sync with backend device.
>
> Previously, DRIVER_OK could be sent twice: 1. when ether layer invokes
> eth_device_init to initialize device; 2. when user configures it with
> different configuration from that of previous.

Yes, but that's wrong. Now only 1) will be taken.

> Since we can only depend on DRIVER_OK notification to sync with backend
> device, we postpone it to virtio_dev_start when everything is settled.

I don't quite understand what you were going to fix here; you don't
state it in the commit log after all. Normally, when everything is
negotiated (when DRIVER_OK is set), we should not set it again, unless
a reset has been happened.

If you look at QEMU, qemu will simply ignore it once vhost has already
started.

	--yliu

Re: [dpdk-dev] DPDK Accelaration Enhancement

[Thomas Monjalon]

2016-12-26 06:19, Ant loves honey:
> I am trying to figure out what it takes to have compression
> support on DPDK such as new PMD driver, additional defines
> and/or API or chipset initialization since the Intel QAT can
> support compression.

You need to solve 2 new things in DPDK:
- introduce a generic compression API
- be able to use a QAT device with a compression driver while
being used with a crypto driver

Interesting work to do :)