From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 7AF655A44; Wed, 11 Jan 2017 05:25:02 +0100 (CET) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP; 10 Jan 2017 20:25:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,345,1477983600"; d="scan'208";a="211972497" Received: from yliu-dev.sh.intel.com ([10.239.67.162]) by fmsmga004.fm.intel.com with ESMTP; 10 Jan 2017 20:25:00 -0800 From: Yuanhan Liu To: dev@dpdk.org Cc: Tan Jianfeng , Wang Zhihong , Yuanhan Liu , Olivier Matz , Maxime Coquelin , "Michael S. Tsirkin" , stable@dpdk.org Date: Wed, 11 Jan 2017 12:27:11 +0800 Message-Id: <1484108832-19907-2-git-send-email-yuanhan.liu@linux.intel.com> X-Mailer: git-send-email 1.9.0 In-Reply-To: <1484108832-19907-1-git-send-email-yuanhan.liu@linux.intel.com> References: <1484108832-19907-1-git-send-email-yuanhan.liu@linux.intel.com> Subject: [dpdk-dev] [PATCH 1/2] net/virtio: fix performance regression due to TSO enabling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2017 04:25:03 -0000 TSO is now enabled, but it's not actually being used by default in a simple L2 forward mode. In such case, we have to zero the virtio net headers, to inform the vhost backend that no offload is being used: hdr->csum_start = 0; hdr->csum_offset = 0; hdr->flags = 0; hdr->gso_type = 0; hdr->gso_size = 0; hdr->hdr_len = 0; Such writes could be very costly; it introduces severe cache issues: The above operations introduce cache write for each packet, which stalls the read operation from the vhost backend. The fact that virtio net header is initiated to zero in PMD driver init stage means that these costly writes are unnecessary and could be avoided: if (hdr->csum_start != 0) hdr->csum_start = 0; And that's what the macro ASSIGN_UNLESS_EQUAL does. With this, the performance drop introduced by TSO enabling is recovered: it could be up to 20% in micro benchmarking. Fixes: 58169a9c8153 ("net/virtio: support Tx checksum offload") Fixes: 696573046e9e ("net/virtio: support TSO") Cc: Olivier Matz Cc: Maxime Coquelin Cc: Michael S. Tsirkin Cc: stable@dpdk.org Signed-off-by: Yuanhan Liu --- drivers/net/virtio/virtio_rxtx.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c index 1e5a6b9..8ec2f1a 100644 --- a/drivers/net/virtio/virtio_rxtx.c +++ b/drivers/net/virtio/virtio_rxtx.c @@ -258,6 +258,12 @@ vtpci_with_feature(hw, VIRTIO_NET_F_HOST_TSO6); } +/* avoid write operation when necessary, to lessen cache issues */ +#define ASSIGN_UNLESS_EQUAL(var, val) do { \ + if ((var) != (val)) \ + (var) = (val); \ +} while (0) + static inline void virtqueue_enqueue_xmit(struct virtnet_tx *txvq, struct rte_mbuf *cookie, uint16_t needed, int use_indirect, int can_push) @@ -337,9 +343,9 @@ break; default: - hdr->csum_start = 0; - hdr->csum_offset = 0; - hdr->flags = 0; + ASSIGN_UNLESS_EQUAL(hdr->csum_start, 0); + ASSIGN_UNLESS_EQUAL(hdr->csum_offset, 0); + ASSIGN_UNLESS_EQUAL(hdr->flags, 0); break; } @@ -355,9 +361,9 @@ cookie->l3_len + cookie->l4_len; } else { - hdr->gso_type = 0; - hdr->gso_size = 0; - hdr->hdr_len = 0; + ASSIGN_UNLESS_EQUAL(hdr->gso_type, 0); + ASSIGN_UNLESS_EQUAL(hdr->gso_size, 0); + ASSIGN_UNLESS_EQUAL(hdr->hdr_len, 0); } } -- 1.9.0