From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9A6DA0520; Thu, 2 Jul 2020 14:21:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4BF3D1D9B6; Thu, 2 Jul 2020 14:21:39 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id E0F6B1D9A6 for ; Thu, 2 Jul 2020 14:21:37 +0200 (CEST) IronPort-SDR: JKyAwUQSLbZgiPntqt7dAn39UwW/tvPM1d507MdIqHJGAZOLdhFRqo2oDKTyaYkOXNJwwCvgyq KzBOXiUbQJdg== X-IronPort-AV: E=McAfee;i="6000,8403,9669"; a="164926023" X-IronPort-AV: E=Sophos;i="5.75,304,1589266800"; d="scan'208";a="164926023" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jul 2020 05:21:36 -0700 IronPort-SDR: 2T9LkHCM8v9a0mbvKXQ4qU5O5VUdOeIiMDegCSVynGHhbYGbubgeTyr14vOUW1RZRD57OM4MAM YbPoQ/GWYxug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,304,1589266800"; d="scan'208";a="281936738" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga006.jf.intel.com with ESMTP; 02 Jul 2020 05:21:36 -0700 Received: from fmsmsx101.amr.corp.intel.com (10.18.124.199) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 2 Jul 2020 05:21:31 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx101.amr.corp.intel.com (10.18.124.199) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 2 Jul 2020 05:21:31 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.172) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 2 Jul 2020 05:21:31 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dE0erYHpoKFluqm+oAmfmlm4iCpvOR84xvhn5Z1Vunqn+hc2Egux1ilOGn6pek5Pt9YTm/ZQtS3YnOcqbHkkvok9tr1VwedsqKNCqyEZlIbcsL/eQhNWFWDlnnJ87LWSSVLpMJnRHr0TF8ZCAPNBe7n2IB47xs2+tAuaI7mJdf+ACD8taxRrhUPyVSSPm21eROm/EIrhPG8ZEubq3t7dbmw0swjKejGrWPHXmJovNV7nKsknbWa1vi2pa6aczOXWg7Z1b30OoAYpjk9YhMjoyctIDP2nuSt3co0grywsGAZDJ+LwJoPgVmRvcHS7oyHX3QDvkR3lDbRcjFYkBC4mhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g3gExbGQ6B/UfGLtrz29S8SXWXtyggTBtzQDPcmnTFw=; b=EcXuqmvKJBiAWD5QkGZPHt0FnjepvPDUrmTPB1HUumGLx4H3x6Y6y77yJy/y2+/0690X2FTvwMZmx4gPC+kvNzNwDMrgiLkTa9Sb+BadWXWd8ysp4qpydjskIRnhG/kTw6i4GOL8RtvKkv7e3izCHYOHmhRl/u2x1gAMYAWNwJL2rteazPt99W48mSlgl5dXypEcu5QIWn9XozSI1twKQ1F/TywNvcfeXr/0Me4+3/5xlNiIXJ/XC513CD2NWfqtDp+UQT93sJsB3vcqwoFP8TKXh0dY3t+q0fenQiEgrHhHU6w2PYqomM3OaLXto1hrIpuKwdEbXgg1nMaiNT75lA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g3gExbGQ6B/UfGLtrz29S8SXWXtyggTBtzQDPcmnTFw=; b=LZzTpP8J+YqPuBNbyIMhVXp/jr5jLIh2EX9mZffaSxCHSyuphY5Q42bWWdont6eLwoHszRzkGJdmezrpOv0B4RatY0RTlItWEp2EFBKFwJNhcRjK3tNM2ExJfBW0PL8kiINplMHhv1+3IK0BRnhI5Y2oY7jl9SJhLiaD4fNY/N4= Received: from BYAPR11MB3735.namprd11.prod.outlook.com (2603:10b6:a03:b4::31) by BYAPR11MB2566.namprd11.prod.outlook.com (2603:10b6:a02:be::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3153.27; Thu, 2 Jul 2020 12:21:28 +0000 Received: from BYAPR11MB3735.namprd11.prod.outlook.com ([fe80::2571:24e3:140b:d78c]) by BYAPR11MB3735.namprd11.prod.outlook.com ([fe80::2571:24e3:140b:d78c%7]) with mapi id 15.20.3153.022; Thu, 2 Jul 2020 12:21:28 +0000 From: "Fu, Patrick" To: "Liu, Yong" , "dev@dpdk.org" , "maxime.coquelin@redhat.com" , "Xia, Chenbo" , "Wang, Zhihong" CC: "Wang, Yinan" , "Jiang, Cheng1" , "Liang, Cunming" Thread-Topic: [dpdk-dev] [PATCH v2 2/2] vhost: introduce async enqueue for split ring Thread-Index: AQHWT4S9n+LPMhcMhk2HQC+nX5zonaj0NWjA Date: Thu, 2 Jul 2020 12:21:28 +0000 Message-ID: References: <1593441849-27306-1-git-send-email-patrick.fu@intel.com> <1593441849-27306-3-git-send-email-patrick.fu@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E63602CC3@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E63602CC3@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.192] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: cb4ff7e2-8fcb-4e01-4216-08d81e827276 x-ms-traffictypediagnostic: BYAPR11MB2566: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0452022BE1 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: xBvz3wmwIpHJGWp/aHewS26DoviuF8ggcR1fWUhJ1o5jIcLAd6nw6bJwPFWuxXpx4e08RGNGQUDIWjiyCwUCIGvptKAQvR1HBNCR31ybty7N5yB2DiBAbU1gU7/YuZhP0p+CBQG1h9PEwEFHgpdS1yZFznRYYYUQvmG1E85m5T4AU7/6HweLehAdJVO/Y+uKnTBXltnXbicJ9NUPSd5xIWXgIIlBNMRHYlAI0hSEoWd1AF36WDY0TraNbVGeWLTf01dPwATQuXwCCCaULj5Qxg0ax7BtbNV+orCb/gH1nnHabxRHo1Mzx2qgaV2zPRawSIgbJtqlot3ovp7XJ9uLVg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3735.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(346002)(396003)(366004)(136003)(39860400002)(9686003)(55016002)(26005)(186003)(6636002)(66946007)(76116006)(64756008)(66476007)(66556008)(66446008)(71200400001)(110136005)(54906003)(316002)(86362001)(7696005)(6506007)(83380400001)(53546011)(4326008)(33656002)(2906002)(107886003)(5660300002)(478600001)(8936002)(52536014); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: Zx3ZiLOzXgAeRGUPKf4BUSaXDncnpGMlv5SKomCiekCLnyKGFDykWnXhx2k1S9HgHNtRjMX2n1iOrvXIWb058n0hIHe4ICZ9mtDplKEJm3gdIFIFR/vkH4PCtTeJZ5hom3cr3Pis42Ou7eovafubdw0YkKzPdX6J6NVAWhX6DaS1Vjk2gK+DndgZtJBLeDXA5uwrmb4S0pVEugsuJDPFvGBip2VtqDWXWXBGd1jt701hBflc8Z8IcY4Y5sUe7gFhVUs3xNaayS2mw8Nm7fIjwLGMUyG52fMyw6caTdM+3IEQKKXVZWTqRkcjB3QZfLBCPw+9XMHvAl+BUrc41NApH56R1GP1m5zZWxXdSILTjOgLd/Zaw36H6YfPOsZQScpMHJVJJfwQpgXvMVZXt/mr1oGq0sPzAhg3pPz4yRS51HYj1e6Zid7M4DHiWNPv2Mi85C5a+q2UMIuk0XmcZ5cfVuB3IFXI8bu9jkhHCUCZkkElbaYHWRRx5xz9tCt8983Q Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3735.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: cb4ff7e2-8fcb-4e01-4216-08d81e827276 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jul 2020 12:21:28.5291 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: RkUHE/03xa+5fhV7tURRZ8ezCNYBsbVhU4MHZSnYLhE6QTtlN3eSu5hhDedj0NMu+cssAP6Lw3ULOlxWg7GwKg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2566 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 2/2] vhost: introduce async enqueue for split ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Thanks Marvin, my comments inline: > -----Original Message----- > From: Liu, Yong > Sent: Wednesday, July 1, 2020 4:51 PM > To: Fu, Patrick ; dev@dpdk.org; > maxime.coquelin@redhat.com; Xia, Chenbo ; Wang, > Zhihong > Cc: Fu, Patrick ; Wang, Yinan > ; Jiang, Cheng1 ; Liang, > Cunming > Subject: RE: [dpdk-dev] [PATCH v2 2/2] vhost: introduce async enqueue for > split ring >=20 > > > > +#define VHOST_ASYNC_BATCH_THRESHOLD 8 > > + >=20 > Not very clear about why batch number is 8. It is better to save it in > rte_vhost_async_features if the value come from hardware requirement. >=20 We are in the progress of benchmarking how this value will have impact to t= he final performance, and we will have a more reasonable manner to handle this macro.=20 > > + > > +static __rte_noinline uint32_t > > +virtio_dev_rx_async_submit_split(struct virtio_net *dev, > > + struct vhost_virtqueue *vq, uint16_t queue_id, > > + struct rte_mbuf **pkts, uint32_t count) { > > + uint32_t pkt_idx =3D 0, pkt_burst_idx =3D 0; > > + uint16_t num_buffers; > > + struct buf_vector buf_vec[BUF_VECTOR_MAX]; > > + uint16_t avail_head, last_idx, shadow_idx; > > + > > + struct rte_vhost_iov_iter *it_pool =3D vq->it_pool; > > + struct iovec *vec_pool =3D vq->vec_pool; > > + struct rte_vhost_async_desc tdes[MAX_PKT_BURST]; > > + struct iovec *src_iovec =3D vec_pool; > > + struct iovec *dst_iovec =3D vec_pool + (VHOST_MAX_ASYNC_VEC >> 1); > > + struct rte_vhost_iov_iter *src_it =3D it_pool; > > + struct rte_vhost_iov_iter *dst_it =3D it_pool + 1; > > + uint16_t n_free_slot, slot_idx; > > + int n_pkts =3D 0; > > + > > + avail_head =3D *((volatile uint16_t *)&vq->avail->idx); > > + last_idx =3D vq->last_avail_idx; > > + shadow_idx =3D vq->shadow_used_idx; > > + > > + /* > > + * The ordering between avail index and > > + * desc reads needs to be enforced. > > + */ > > + rte_smp_rmb(); > > + > > + rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - > > +1)]); > > + > > + for (pkt_idx =3D 0; pkt_idx < count; pkt_idx++) { > > + uint32_t pkt_len =3D pkts[pkt_idx]->pkt_len + dev->vhost_hlen; > > + uint16_t nr_vec =3D 0; > > + > > + if (unlikely(reserve_avail_buf_split(dev, vq, > > + pkt_len, buf_vec, > > &num_buffers, > > + avail_head, &nr_vec) < 0)) { > > + VHOST_LOG_DATA(DEBUG, > > + "(%d) failed to get enough desc from > > vring\n", > > + dev->vid); > > + vq->shadow_used_idx -=3D num_buffers; > > + break; > > + } > > + > > + VHOST_LOG_DATA(DEBUG, "(%d) current index %d | end > > index %d\n", > > + dev->vid, vq->last_avail_idx, > > + vq->last_avail_idx + num_buffers); > > + > > + if (async_mbuf_to_desc(dev, vq, pkts[pkt_idx], > > + buf_vec, nr_vec, num_buffers, > > + src_iovec, dst_iovec, src_it, dst_it) < 0) { > > + vq->shadow_used_idx -=3D num_buffers; > > + break; > > + } > > + > > + slot_idx =3D (vq->async_pkts_idx + pkt_idx) & (vq->size - 1); > > + if (src_it->count) { > > + async_fill_des(&tdes[pkt_burst_idx], src_it, dst_it); > > + pkt_burst_idx++; > > + vq->async_pending_info[slot_idx] =3D > > + num_buffers | (src_it->nr_segs << 16); > > + src_iovec +=3D src_it->nr_segs; > > + dst_iovec +=3D dst_it->nr_segs; > > + src_it +=3D 2; > > + dst_it +=3D 2; >=20 > Patrick, > In my understanding, nr_segs type definition can follow nr_vec type > definition (uint16_t). By that can short the data saved in async_pkts_pen= ding > from 64bit to 32bit. > Since those information will be used in datapath, the smaller size will g= et the > better perf. >=20 > It is better to replace integer 2 with macro. >=20 will update the code as you suggested. Thanks, Patrick