From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C371CA0613 for ; Mon, 23 Sep 2019 13:09:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BC7B81BEB5; Mon, 23 Sep 2019 13:09:41 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130079.outbound.protection.outlook.com [40.107.13.79]) by dpdk.org (Postfix) with ESMTP id DF3221BE87 for ; Mon, 23 Sep 2019 13:09:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gShd5Uxy8BIFsYuyjgfGC4O1KAH9EH0Yc2VIb3xViNA=; b=w5Q8gVHaL3t3SgzUzblmQf63K4iU9OKStKSV5kkK8Qvdk+QY4wFHnemABDZjY2J/0rw+4zb0qokO+Hm/2ulCcozC/CVhk4aiDAa9RZovNpFmsSmQZve0hNYp18FWvKcX25rrk0e05o2+zIuh5wqbz04t5PjBazDfYMZpwA8vrOs= Received: from VI1PR08CA0195.eurprd08.prod.outlook.com (2603:10a6:800:d2::25) by DB6PR0801MB1688.eurprd08.prod.outlook.com (2603:10a6:4:3a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.18; Mon, 23 Sep 2019 11:09:38 +0000 Received: from DB5EUR03FT018.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::204) by VI1PR08CA0195.outlook.office365.com (2603:10a6:800:d2::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.21 via Frontend Transport; Mon, 23 Sep 2019 11:09:38 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT018.mail.protection.outlook.com (10.152.20.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20 via Frontend Transport; Mon, 23 Sep 2019 11:09:37 +0000 Received: ("Tessian outbound 55d20e99e8e2:v31"); Mon, 23 Sep 2019 11:09:04 +0000 X-CR-MTA-TID: 64aa7808 Received: from f354dd180337.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.10.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id F7B5BC95-1E62-47AB-A294-6D5DDBD011A4.1; Mon, 23 Sep 2019 11:08:59 +0000 Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-db5eur03lp2052.outbound.protection.outlook.com [104.47.10.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f354dd180337.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 23 Sep 2019 11:08:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=elG91Rk+leAuja9O4PYqMnoSaVllvwvPf+BZmuiIbmK7Wpt+tabQY19nn4UWPhEDhUSejY+SEHoL4gboNcAczYdJ7yjW+rF7gqV08YLdf8ur/C77bvLO3GvZ6OuZi8WbTw8+WgSfoZFIB5M97I1V9Z5Sh6vNST/u9SyLKQiXMXFcBhDku2AJnDxO/L5NXUjOmKS+OEDk6rXV9/z24Pu8JKXMdHJ1U+E+6ReUuQGCLN4X5QagqyP5BhHm9Uz8mBeLdaLZIhs6/Z6cKljhgAbY4qHQwGratoKtAbn+tUEa5HVJGsjm4stYnNJEevCDVddX64qiColS7rs6Ju3NoF77qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gShd5Uxy8BIFsYuyjgfGC4O1KAH9EH0Yc2VIb3xViNA=; b=KFOSAN94pKUCyOcAfIfgdfQSW9XlTfeeDNpaGzD/G+AduNuqNxcZya9L8u4YWOr5d3Lq7oFjCZkwv3C7Vd5SSQGYpHtkH3fN/IkiROgAv/q/GaUuMbzYh8k91qkdzzqK+TUB6C/cgczIopjv2AsVtBVfvdRcKEJ1mQHXGoEgj7aBqwbucHRWeorNwRibwzpyhdeFQXS6BaJzRGpNb5szTZsgpMa2JlwLyqnX/3k96KXNrKFAjpQYKvmcesJzJO/R1NyB5+mB6Xp1dyE+49B8WBMn6htJn1E9/VLJGtEjGYQMuJ4QqPGcle8sWkVCue4HT6l1PdWVaPNrpnioJTo7eQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gShd5Uxy8BIFsYuyjgfGC4O1KAH9EH0Yc2VIb3xViNA=; b=w5Q8gVHaL3t3SgzUzblmQf63K4iU9OKStKSV5kkK8Qvdk+QY4wFHnemABDZjY2J/0rw+4zb0qokO+Hm/2ulCcozC/CVhk4aiDAa9RZovNpFmsSmQZve0hNYp18FWvKcX25rrk0e05o2+zIuh5wqbz04t5PjBazDfYMZpwA8vrOs= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (52.133.244.200) by VI1PR08MB4430.eurprd08.prod.outlook.com (20.179.27.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20; Mon, 23 Sep 2019 11:08:57 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707%3]) with mapi id 15.20.2284.023; Mon, 23 Sep 2019 11:08:56 +0000 From: "Gavin Hu (Arm Technology China)" To: Marvin Liu , "maxime.coquelin@redhat.com" , "tiwei.bie@intel.com" , "zhihong.wang@intel.com" CC: "dev@dpdk.org" , nd Thread-Topic: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue function for packed ring Thread-Index: AQHVbsgy8n4dxJX8MUO7ryE7+kcY6ac5EPEQ Date: Mon, 23 Sep 2019 11:08:56 +0000 Message-ID: References: <20190905161421.55981-2-yong.liu@intel.com> <20190919163643.24130-1-yong.liu@intel.com> <20190919163643.24130-4-yong.liu@intel.com> In-Reply-To: <20190919163643.24130-4-yong.liu@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: d487a6f7-0b25-4f90-a1c3-ad7ad4036770.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: b3e57be7-8ca1-4a0e-e2d1-08d7401685a7 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR08MB4430; X-MS-TrafficTypeDiagnostic: VI1PR08MB4430:|DB6PR0801MB1688: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:4303;OLM:4303; x-forefront-prvs: 0169092318 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(136003)(376002)(39860400002)(346002)(366004)(13464003)(189003)(199004)(476003)(25786009)(33656002)(53546011)(9686003)(55236004)(2201001)(7696005)(52536014)(66556008)(66476007)(6246003)(66446008)(55016002)(229853002)(6506007)(5660300002)(2501003)(81156014)(81166006)(66066001)(6436002)(76116006)(86362001)(4326008)(186003)(64756008)(99286004)(26005)(446003)(11346002)(486006)(7736002)(305945005)(74316002)(478600001)(102836004)(256004)(14444005)(2906002)(66946007)(54906003)(6116002)(110136005)(8936002)(3846002)(76176011)(14454004)(316002)(71200400001)(71190400001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB4430; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: YdCSSiXjkjqKCDkNkVz4EphiM6dEFlR0NOzgVLJ3qvLe+I/+dmtz0CeoOAtc7Pc+6fEPx9a6j9o40jR6ypsYHJh8D8nzkVjVz+Hdt2HAoAe/MDcmcL2iadlpqiBGtifCwi/dKlOlt8cCZU5mKa5M6KkWu6l9IgNNIeZ/tSTqsFPo9lEphEKfYWZ/VkF2UWo74N0zwICGDXikYe99izGRy/wE89l0MXYkQri0BFPB0MbqP96SvzBKJSupVsQxRjCJsHa8SpTd/53hioL5b6rnmJPGVzIy5DXKYrVTUV7ILbyKk8FDXBDKmTNybfVZrCpXVoRVB56Ry/aAnZd2WpfNXZKO4xdAOrxCARoppUpTy3LZjmgjWlzaTK/xF+SXbhmJaw4p45PaScFraX/gu2X92sHVjgM1sAY5cycSilJOzc0= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4430 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT018.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(396003)(39860400002)(376002)(346002)(189003)(199004)(13464003)(186003)(81166006)(81156014)(50466002)(229853002)(52536014)(54906003)(4326008)(76130400001)(70206006)(33656002)(63350400001)(22756006)(11346002)(26826003)(446003)(47776003)(99286004)(76176011)(66066001)(110136005)(14454004)(7696005)(26005)(25786009)(486006)(70586007)(316002)(8746002)(478600001)(8936002)(336012)(2501003)(55016002)(46406003)(6246003)(476003)(6116002)(2201001)(7736002)(9686003)(126002)(5660300002)(86362001)(3846002)(102836004)(23726003)(53546011)(14444005)(74316002)(305945005)(356004)(97756001)(6506007)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0801MB1688; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 4ba47b13-4a20-4e99-b119-08d740166daa X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DB6PR0801MB1688; NoDisclaimer: True X-Forefront-PRVS: 0169092318 X-Microsoft-Antispam-Message-Info: csw38luQ9q4StzuNQVY3t0vfPfbA8BqJgERSjvd6Obs61bGcU60bqXuLFkP4o+4JAhKFOpjxn+MdaPJRxVILGmOGh08LRIj/yPNuoHT6tbIfmUjcABCALdlZpsxWYoI0mSDkg5kqEqJKgtMh/MUVPal3AhPO+7O6XhaX2z0/2Bap0E9mrPtFR5sOWZJ8auuUSVfByYY8G2Cvz4J9+GHGnD9YGBlsE7CuRwzwf8U9pSidRd1YXJMtQ55tGUSNxibcr0gVeFc7r6q9zCjmjCN3xA83Vvzovh0LQd5GpmeEd6i/aQR8YvNufBAUcizkONQ2MlBaeRgEGtJmyd2DWyCHcGZvv1FYz5MaOMjqxnDrqxSxzApb9tNDo3YdIvFBxF978yhOBw5ma6Xt97yFIIpEZW3T1KTayWQHe9kfdbLp9XE= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2019 11:09:37.1564 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b3e57be7-8ca1-4a0e-e2d1-08d7401685a7 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1688 Subject: Re: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue function for packed ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Marvin, Is it possible to vectorize the processing?=20 Other comments inline: /Gavin > -----Original Message----- > From: dev On Behalf Of Marvin Liu > Sent: Friday, September 20, 2019 12:37 AM > To: maxime.coquelin@redhat.com; tiwei.bie@intel.com; > zhihong.wang@intel.com > Cc: dev@dpdk.org; Marvin Liu > Subject: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue function fo= r > packed ring >=20 > Burst enqueue function will first check whether descriptors are cache > aligned. It will also check prerequisites in the beginning. Burst > enqueue function not support chained mbufs, single packet enqueue > function will handle it. >=20 > Signed-off-by: Marvin Liu >=20 > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > index 5074226f0..67889c80a 100644 > --- a/lib/librte_vhost/vhost.h > +++ b/lib/librte_vhost/vhost.h > @@ -39,6 +39,9 @@ >=20 > #define VHOST_LOG_CACHE_NR 32 >=20 > +#define PACKED_DESCS_BURST (RTE_CACHE_LINE_SIZE / \ > + sizeof(struct vring_packed_desc)) > + > #ifdef SUPPORT_GCC_UNROLL_PRAGMA > #define PRAGMA_PARAM "GCC unroll 4" > #endif > @@ -57,6 +60,8 @@ > #define UNROLL_PRAGMA(param) do {} while(0); > #endif >=20 > +#define PACKED_BURST_MASK (PACKED_DESCS_BURST - 1) > + > /** > * Structure contains buffer address, length and descriptor index > * from vring to do scatter RX. > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.= c > index 2b5c47145..c664b27c5 100644 > --- a/lib/librte_vhost/virtio_net.c > +++ b/lib/librte_vhost/virtio_net.c > @@ -895,6 +895,84 @@ virtio_dev_rx_split(struct virtio_net *dev, struct > vhost_virtqueue *vq, > return pkt_idx; > } >=20 > +static __rte_unused __rte_always_inline int I remember "__rte_always_inline" should start at the first and separate lin= e, otherwise you will get a style issue.=20 /Gavin > +virtio_dev_rx_burst_packed(struct virtio_net *dev, struct vhost_virtqueu= e > *vq, > + struct rte_mbuf **pkts) > +{ > + bool wrap_counter =3D vq->avail_wrap_counter; > + struct vring_packed_desc *descs =3D vq->desc_packed; > + uint16_t avail_idx =3D vq->last_avail_idx; > + > + uint64_t desc_addrs[PACKED_DESCS_BURST]; > + struct virtio_net_hdr_mrg_rxbuf *hdrs[PACKED_DESCS_BURST]; > + uint32_t buf_offset =3D dev->vhost_hlen; > + uint64_t lens[PACKED_DESCS_BURST]; > + > + uint16_t i; > + > + if (unlikely(avail_idx & PACKED_BURST_MASK)) > + return -1; > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > + if (unlikely(pkts[i]->next !=3D NULL)) > + return -1; > + if (unlikely(!desc_is_avail(&descs[avail_idx + i], > + wrap_counter))) > + return -1; > + } > + > + rte_smp_rmb(); > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > + lens[i] =3D descs[avail_idx + i].len; Looks like the code is a strong candidate for vectorization.=20 > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > + if (unlikely(pkts[i]->pkt_len > (lens[i] - buf_offset))) > + return -1; > + } > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > + desc_addrs[i] =3D vhost_iova_to_vva(dev, vq, > + descs[avail_idx + i].addr, > + &lens[i], > + VHOST_ACCESS_RW); > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > + if (unlikely(lens[i] !=3D descs[avail_idx + i].len)) > + return -1; > + } > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > + rte_prefetch0((void *)(uintptr_t)desc_addrs[i]); > + hdrs[i] =3D (struct virtio_net_hdr_mrg_rxbuf *)desc_addrs[i]; > + lens[i] =3D pkts[i]->pkt_len + dev->vhost_hlen; > + } > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > + virtio_enqueue_offload(pkts[i], &hdrs[i]->hdr); > + A store barrier here is missing, last_avail_idx may be observed before the = above enqueue completion on weak memory order architectures. For x86, a compiler barrier is also required.=20 > + vq->last_avail_idx +=3D PACKED_DESCS_BURST; > + if (vq->last_avail_idx >=3D vq->size) { > + vq->last_avail_idx -=3D vq->size; > + vq->avail_wrap_counter ^=3D 1; > + } > + > + UNROLL_PRAGMA(PRAGMA_PARAM) > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > + rte_memcpy((void *)(uintptr_t)(desc_addrs[i] + buf_offset), > + rte_pktmbuf_mtod_offset(pkts[i], void *, 0), > + pkts[i]->pkt_len); > + } > + > + return 0; > +} > + > static __rte_unused int16_t > virtio_dev_rx_single_packed(struct virtio_net *dev, struct vhost_virtque= ue > *vq, > struct rte_mbuf *pkt) > -- > 2.17.1