From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id E7D4C8D3D for ; Tue, 27 Oct 2015 02:44:13 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP; 26 Oct 2015 18:44:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,202,1444719600"; d="scan'208";a="803793086" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by orsmga001.jf.intel.com with ESMTP; 26 Oct 2015 18:44:14 -0700 Received: from fmsmsx117.amr.corp.intel.com (10.18.116.17) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 26 Oct 2015 18:44:12 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx117.amr.corp.intel.com (10.18.116.17) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 26 Oct 2015 18:44:12 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.204]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.96]) with mapi id 14.03.0248.002; Tue, 27 Oct 2015 09:44:10 +0800 From: "Tan, Jianfeng" To: "Xie, Huawei" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v5 0/7] virtio ring layout optimization and simple rx/tx processing Thread-Index: AQHRDzrL8odiH1QQUEeQhR1npCtxAJ59hKZA Date: Tue, 27 Oct 2015 01:44:09 +0000 Message-ID: References: <1443537953-23917-1-git-send-email-huawei.xie@intel.com> <1445787304-18267-1-git-send-email-huawei.xie@intel.com> In-Reply-To: <1445787304-18267-1-git-send-email-huawei.xie@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v5 0/7] virtio ring layout optimization and simple rx/tx processing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Oct 2015 01:44:14 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Huawei Xie > Sent: Sunday, October 25, 2015 11:35 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v5 0/7] virtio ring layout optimization and si= mple > rx/tx processing >=20 > Changes in v5: > - Call __rte_pktmbuf_prefree_seg to check refcnt when free mbufs >=20 > Changes in v4: > - Fix the error in virtio tx ring layout ascii chart in the commit messag= e > - Move virtio_xmit_cleanup ahead to free descriptors earlier > - Test merge-able feature when select simple rx/tx functions >=20 > Changes in v3: > - Remove unnecessary NULL test for rte_free > - Remove unnecessary assign of local var after free > - Remove return at the end of void function > - Remove always_inline attribute for virtio_xmit_cleanup > - Reword some commit messages > - Add TODO in the commit message of simple tx patch >=20 > Changes in v2: > - Remove the configure macro > - Enable simple R/TX processing when user specifies simple txq flags > - Reword some comments and commit messages >=20 > In DPDK based switching enviroment, mostly vhost runs on a dedicated core > while virtio processing in guest VMs runs on other different cores. > Take RX for example, with generic implementation, for each guest buffer, > a) virtio driver allocates a descriptor from free descriptor list > b) modify the entry of avail ring to point to allocated descriptor > c) after packet is received, free the descriptor >=20 > When vhost fetches the avail ring, it need to fetch the modified L1 cache > from virtio core, which is a heavy cost in current CPU implementation. >=20 > This idea of this optimization is: > allocate the fixed descriptor for each entry of avail ring, so avail = ring will > always be the same during the run. > This removes L1M cache transfer from virtio core to vhost core for avail = ring. > (Note we couldn't avoid the cache transfer for descriptors). > Besides, descriptor allocation and free operation is eliminated. > This also makes vector procesing possible to further accelerate the > processing. >=20 > This is the layout for the avail ring(take 256 ring entries for example),= with > each entry pointing to the descriptor with the same index. > avail > idx > + > | > +----+----+---+-------------+------+ > | 0 | 1 | 2 | ... | 254 | 255 | avail ring > +-+--+-+--+-+-+---------+---+--+---+ > | | | | | | > | | | | | | > v v v | v v > +-+--+-+--+-+-+---------+---+--+---+ > | 0 | 1 | 2 | ... | 254 | 255 | desc ring > +----+----+---+-------------+------+ > | > | > +----+----+---+-------------+------+ > | 0 | 1 | 2 | | 254 | 255 | used ring > +----+----+---+-------------+------+ > | > + >=20 > This is the ring layout for TX. > As we need one virtio header for each xmit packet, we have 128 slots > available. >=20 > ++ > || > || > +-----+-----+-----+--------------+------+------+------+ > | 0 | 1 | ... | 127 || 128 | 129 | ... | 255 | avail ring > +--+--+--+--+-----+---+------+---+--+---+------+--+---+ > | | | || | | | > v v v || v v v > +--+--+--+--+-----+---+------+---+--+---+------+--+---+ > | 127 | 128 | ... | 255 || 127 | 128 | ... | 255 | desc ring for v= irtio_net_hdr > +--+--+--+--+-----+---+------+---+--+---+------+--+---+ > | | | || | | | > v v v || v v v > +--+--+--+--+-----+---+------+---+--+---+------+--+---+ > | 0 | 1 | ... | 127 || 0 | 1 | ... | 127 | desc ring for t= x dat > +-----+-----+-----+--------------+------+------+------+ > || > || > ++ >=20 >=20 > Performance boost could be observed only if the virtio backend isn't the > bottleneck or in VM2VM case. > There are also several vhost optimization patches to be submitted later. >=20 >=20 > Huawei Xie (7): > virtio: add virtio_rxtx.h header file > virtio: add software rx ring, fake_buf into virtqueue > virtio: rx/tx ring layout optimization > virtio: fill RX avail ring with blank mbufs > virtio: virtio vec rx > virtio: simple tx routine > virtio: pick simple rx/tx func >=20 > drivers/net/virtio/Makefile | 2 +- > drivers/net/virtio/virtio_ethdev.c | 12 +- > drivers/net/virtio/virtio_ethdev.h | 5 + > drivers/net/virtio/virtio_rxtx.c | 56 ++++- > drivers/net/virtio/virtio_rxtx.h | 39 +++ > drivers/net/virtio/virtio_rxtx_simple.c | 414 > ++++++++++++++++++++++++++++++++ > drivers/net/virtio/virtqueue.h | 5 + > 7 files changed, 529 insertions(+), 4 deletions(-) create mode 100644 > drivers/net/virtio/virtio_rxtx.h create mode 100644 > drivers/net/virtio/virtio_rxtx_simple.c >=20 > -- > 1.8.1.4 Acked-by Jianfeng Tan Thanks, Jianfeng