From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by dpdk.org (Postfix) with ESMTP id 13D1414EC for ; Fri, 18 Jan 2019 18:56:10 +0100 (CET) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id x0IHsO7L021431; Fri, 18 Jan 2019 17:56:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=mime-version : message-id : date : from : sender : to : subject : references : in-reply-to : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=0cvfl5r8jp6adyTIyAMrO83H8kBOO6W99YDrsMxr4YY=; b=QWpFzmPEMP1BGVIfOoBHcoeZHdIXmV6+CDtGTpxPrG8+1xPX04xkkKnz6GSoRXlURBis YCQMOE8JJ/K4bizHCe8YeBgjOY/m5URmBiyfZWGd4u56dXNa1A0BsgoEsjmxzXVxSnwD DAif4QajRSho/AFjXT/OupjmlJYMjUjMCwq0Tj3q7BkLnY1VKiglJboxERMyPsL1hRFu ZPHGdff7Sf5Q0oQv5MMsR+FxsPBBbil1Uh3B1TnheAExvxhl3l45zCvj7FpykgrSZIK3 p1WrpQjzYqORVPJ27NA1LQBKfL81A0UxmOx7JzgqEOyyvtol0u+kCi+0kcu4slYrG8ip dA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2pybjp6w60-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 18 Jan 2019 17:56:09 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x0IHu3cX030806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 18 Jan 2019 17:56:04 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x0IHu3BI017221; Fri, 18 Jan 2019 17:56:03 GMT MIME-Version: 1.0 Message-ID: Date: Fri, 18 Jan 2019 09:55:58 -0800 (PST) From: Changchun Zhang Sender: Changchun Zhang To: "Trahe, Fiona" , "Pathak, Pravin" , users@dpdk.org References: <03fd164b-112b-4e44-a5b0-15c6e3703662@default> <348A99DA5F5B7549AA880327E580B435896CD08F@IRSMSX101.ger.corp.intel.com> <168A68C163D584429EF02A476D5274424DEA9B7C@FMSMSX108.amr.corp.intel.com> <5e87ae90-94b9-4e85-9172-46b95365ec36@default> <348A99DA5F5B7549AA880327E580B435896CD528@IRSMSX101.ger.corp.intel.com> <400475b7-869e-4281-9d31-058f96957fe1@default> <348A99DA5F5B7549AA880327E580B435896CD61A@IRSMSX101.ger.corp.intel.com> In-Reply-To: <348A99DA5F5B7549AA880327E580B435896CD61A@IRSMSX101.ger.corp.intel.com> X-Priority: 3 X-Mailer: Oracle Beehive Extensions for Outlook 2.0.1.9.1 (1003210) [OL 15.0.5093.0 (x86)] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9140 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1901180129 Subject: Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2019 17:56:11 -0000 Thanks! Changchun (Alex) -----Original Message----- From: Trahe, Fiona [mailto:fiona.trahe@intel.com]=20 Sent: Friday, January 18, 2019 11:58 AM To: Changchun Zhang ; Pathak, Pravin ; users@dpdk.org Cc: Trahe, Fiona Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPD= K Hi Alex, > -----Original Message----- > From: Changchun Zhang [mailto:changchun.zhang@oracle.com] > Sent: Friday, January 18, 2019 4:42 PM > To: Trahe, Fiona ; Pathak, Pravin=20 > ; users@dpdk.org > Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD=20 > in DPDK >=20 >=20 >=20 > Thanks! > Changchun (Alex) >=20 >=20 > -----Original Message----- > From: Trahe, Fiona [mailto:fiona.trahe@intel.com] > Sent: Friday, January 18, 2019 11:26 AM > To: Changchun Zhang ; Pathak, Pravin=20 > ; users@dpdk.org > Cc: Trahe, Fiona > Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD=20 > in DPDK >=20 > Hi Alex, > > [changchun] In the same thread, but how about to dequeuer at the=20 > > beginning of the thread each time, if data presents then processing=20 > > them, if no data just do other work, and equeue the packets at some tim= e but does not wait. > > For example: > > While(1) > > { > > =09Nb_ops =3D dequeuer(); > > =09If(nb_ops > ) > > { > > process_dequeued_data(); > > continue; > > } > > > > Other_work(); > > If(ipsec) > > Enqueuer(); > > } > > Does it make sense? > [Fiona] It can, though on the first loop ro after a queit time youll=20 > proably get very few back on first and second dequeue as It'll be=20 > called immediately after the enqueue. Once it gets busy that could be=20 > ok though [changchun] Thank you Fiona. One more question, as you said,=20 > enqueuer/dequeue should be called within the same thread. Why? Is it=20 > because the other thread(lcore 1) cannot dequeuer the processed data=20 > from other thread(lcore 2)? But as the cryptograph device lib doc=20 > says, "it is howerver possible to use a different logical core to dequeue= r an operation on a queue pair from the logical core which it was enqueued = on". Looking forward to more details. It's because the QAT inflight counter would be incremented and decremented = by both threads so would need to be an atomic. It used to be atomic until 1= 7.11 release but we got a good reduction in offload cycle-count by replacin= g this with a normal variable and as all the feedback we got was that appli= cations were not using in pipeline mode we decided to trade off this limita= tion for the added performance. The limitation is documented here: https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__doc.dpdk.org_guides_c= ryptodevs_qat.html&d=3DDwIFAg&c=3DRoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_J= nE&r=3DVCO7jqQFACS5pfBnqRHRln4bZ3htFERYZ3PkX0ytpns&m=3DBp2X1mxAJ_Eg1kB1t91m= gFDUfIrr8Vl-2Nxq_9RLxrI&s=3D2MPxBDAAKTRMsgE6mrw0cbGX8C4oFcFstJWzpEp8opI&e= =3D You can look at code before 17.11 release to see the difference. [changchun] Many thanks! So from this limitation, we can conclude that Lco= re can only dequeue the QAT queue which was enqueued by itself, right. If s= o, then the Crypto device lib doc may be a little misleading, at least some= notes should be put there.