From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFBA945AA7; Fri, 4 Oct 2024 04:22:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7D970402DD; Fri, 4 Oct 2024 04:22:51 +0200 (CEST) Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by mails.dpdk.org (Postfix) with ESMTP id DE650402C9 for ; Fri, 4 Oct 2024 04:22:49 +0200 (CEST) Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-207115e3056so14772195ad.2 for ; Thu, 03 Oct 2024 19:22:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1728008569; x=1728613369; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=NgYVoXFvq9BRkl7Arj7LVlJp9IbkaKm4ib9yqbZtJ4E=; b=AlJPNCjhJh/Oqy4VS+bEQ/xVcucplwYMA41MfpDtihPva4JXxTuiKcbSa2hzt0peJ4 ED/wC1JRsGk2l0kyQevTjlAer+3WfqOtuObkgRz/Mr/sTknaYvK3Gzj4d5FL99kY0786 8OdbwSoh4DiVnwANgMLwaBm22oejj/dO4XXEbEJy1O8/SSGX3SBamg9BLhTYQebLohcY ayn4jxn4bB7BkJ7mNLk85NDa1nsMR3Da65o/+A7zvjYz6V+Ymqzj0dg2v0V8obhbLaOa 9HpesYW7DkpwDdXA4tIN3ZDrnjJHIYE5nRnadECF3sDAdePltfVO1DEwD3b5PFVw+mv2 9VGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728008569; x=1728613369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NgYVoXFvq9BRkl7Arj7LVlJp9IbkaKm4ib9yqbZtJ4E=; b=NJAn8HH2enKONPJpAC6h5eURRsC/XBPq6KGPF33bAHRrHHa/h4/lQVW61wm78l51ui TdWedPWXfyZL8CzvXjuCFi1tYkHFZQWLrO/baF4cI9EjethdNb8nVK3JauZsnHyj24hm E8mJrnzSobpjVQoVvVkvU9La5chnlz+v1tRpAl4WvH5iVN7Ph1yanWI8fpHtpGPhch/M gab3DTLKTNsskT7JBi/fwNk4ZAzAzfljPGKESHkMbUjnn0bs1CC0JQKvWImDbiu4MzT5 u3dvz7hmxBvqr4Dkz9BdVK4r+H9Xobn2xR9kXKa5LCdLSQHBgR7Y9depXrxn37FIj0Ub yr1g== X-Forwarded-Encrypted: i=1; AJvYcCVaTJTaiqtFr4UAA+XsJjHaOO7i3FMnpWCiKimefOOb7WMkCjpxFV7mdwpaYR3dkAcM8QM=@dpdk.org X-Gm-Message-State: AOJu0YwQRDEwBK1MGXXCkvN/8N/Syq3eLzeSNmXRQlqzU9N8Iud+jF8c B5IwUx1ro+GSSiyIVEzlW9zhVPGl7o98R2tICdEHxub4r9NHgXNj3Kpl+s53njI= X-Google-Smtp-Source: AGHT+IExAJ0ZOg4t0zDIkA4yI2Rl6n/NjGi8LxyFb3TKeiHUB0l/freErN+C5AEdujRCXOtTh3WFwA== X-Received: by 2002:a17:902:e806:b0:20b:7ed8:3990 with SMTP id d9443c01a7336-20bfdf81003mr18402865ad.12.1728008568965; Thu, 03 Oct 2024 19:22:48 -0700 (PDT) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20beead2013sm15375675ad.56.2024.10.03.19.22.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 19:22:48 -0700 (PDT) Date: Thu, 3 Oct 2024 19:22:46 -0700 From: Stephen Hemminger To: Wenwu Ma Cc: maxime.coquelin@redhat.com, chenbo.xia@intel.com, dev@dpdk.org, sunil.pai.g@intel.com, jiayu.hu@intel.com, yinan.wang@intel.com, xingguang.he@intel.com, xuan.ding@intel.com, cheng1.jiang@intel.com, yuanx.wang@intel.com Subject: Re: [PATCH v4] vhost: support CPU copy for small packets Message-ID: <20241003192246.4a83f3cd@hermes.local> In-Reply-To: <20220829005658.84590-1-wenwux.ma@intel.com> References: <20220812064517.272530-1-wenwux.ma@intel.com> <20220829005658.84590-1-wenwux.ma@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, 29 Aug 2022 08:56:58 +0800 Wenwu Ma wrote: > Offloading small packets to DMA degrades throughput 10%~20%, > and this is because DMA offloading is not free and DMA is not > good at processing small packets. In addition, control plane > packets are usually small, and assign those packets to DMA will > significantly increase latency, which may cause timeout like > TCP handshake packets. Therefore, this patch use CPU to perform > small copies in vhost. > > Signed-off-by: Wenwu Ma > --- > v4: > * fix coding style issues > v3: > * compare threshold with entire packet length > v2: > * fix CI build error > --- > lib/vhost/vhost.h | 7 ++-- > lib/vhost/virtio_net.c | 73 +++++++++++++++++++++++++++++++++--------- > 2 files changed, 62 insertions(+), 18 deletions(-) > > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h > index 40fac3b7c6..8a7d90f737 100644 > --- a/lib/vhost/vhost.h > +++ b/lib/vhost/vhost.h > @@ -142,8 +142,10 @@ struct virtqueue_stats { > * iovec > */ > struct vhost_iovec { > - void *src_addr; > - void *dst_addr; > + void *src_iov_addr; > + void *dst_iov_addr; > + void *src_virt_addr; > + void *dst_virt_addr; > size_t len; > }; > > @@ -155,6 +157,7 @@ struct vhost_iov_iter { > struct vhost_iovec *iov; > /** number of iovec in this iterator */ > unsigned long nr_segs; > + unsigned long nr_len; > }; > > struct async_dma_vchan_info { > diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c > index 35fa4670fd..cf796183a0 100644 > --- a/lib/vhost/virtio_net.c > +++ b/lib/vhost/virtio_net.c > @@ -26,6 +26,8 @@ > > #define MAX_BATCH_LEN 256 > > +#define CPU_COPY_THRESHOLD_LEN 256 Good idea. This heuristic matches what Linux network drivers do to avoid creating large buffers from small packets. Patch no longer applies cleanly to main branch (after 2 years). Also the copy threshold should be configurable via dev_args to allow for testing and CPU differences.