From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFDA1A0353 for ; Thu, 24 Feb 2022 04:23:59 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 96AC541163; Thu, 24 Feb 2022 04:23:59 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id C5F8240DF6 for ; Thu, 24 Feb 2022 04:23:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645673037; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CLYmPytNi8Y5ncMBx5Mf9J1ouptftAJw22Im2XIZ2e8=; b=Hi6Y1ySzW5FlliZUAolR6F+K1CcU7Mfw2JfoViHPrQCJQ2t4H7/5UOhDyx+4ejrBhFj9ow A2Yt/MxF10pwub57RTBs2q3od1nvM4rc/3tI2nVX60PjGGgWDembrIMkMZRZx4yGdrwOrk eyQ+EPPSrlOnROpXfPc5tsQs4WLiREA= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-JaE3tXHEMs6rtsS5DAjVLA-1; Wed, 23 Feb 2022 22:23:53 -0500 X-MC-Unique: JaE3tXHEMs6rtsS5DAjVLA-1 Received: by mail-lf1-f70.google.com with SMTP id c25-20020a056512325900b0043fc8f2e1f6so79968lfr.6 for ; Wed, 23 Feb 2022 19:23:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=CLYmPytNi8Y5ncMBx5Mf9J1ouptftAJw22Im2XIZ2e8=; b=LuLbrJiYhSnsWUD1qI6PEvD6ZlhK3Xh17J1jyJNauBHpHy7v/Th6GwvZD5lf5TQb7b o2QIkzzHfQsbGDKV/9XOzU3oqQ/jnYT23MPR1bSfKydj7ydy+HTgzIe+Ptr/si59do+e 6mrr6Nm1dNhvVh88ls6PnOzr/q7ITqpUyS1WN8aevC3YuXJoKwizYJyrvoVvlMCdIqEy uFHJKOFxufkfavrx5w5fxzhAWjdau+eydCKS/SOIWLW16iyZrehgV4TnNIBrlwSZEotH SqdckBZwHLj4bkxa+LEJpgKFE4MeK81SgyIKZ+VRI+2ENovIv4U0NcLY4IJ4PWl7JyAq pvyA== X-Gm-Message-State: AOAM531kpWXc/y+gcWk7L4CfNDAu4mGF7YBsMP58GmXOklfGi0LcGKSZ oU2ovLeyPZLPYwU76rQ5gfXORvtMs9c1NdvRMqAbp3F0UQkHpbmRrgEUp77UQLAUpFUjdUyqyYd dr148vH8kSKoUZ+U92rM0cQ== X-Received: by 2002:a05:6512:3147:b0:443:323d:179d with SMTP id s7-20020a056512314700b00443323d179dmr579180lfi.98.1645673031412; Wed, 23 Feb 2022 19:23:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJzUe5dwgs4Zt8RJjJa9dYBCHoUjub4YEZ6koNq4tb1tfp+jvAlBT28t478zjTiwSykr/+BPx9EVwpRaN2ShvWA= X-Received: by 2002:a05:6512:3147:b0:443:323d:179d with SMTP id s7-20020a056512314700b00443323d179dmr579169lfi.98.1645673031202; Wed, 23 Feb 2022 19:23:51 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Jason Wang Date: Thu, 24 Feb 2022 11:23:39 +0800 Message-ID: Subject: Re: Question about the sndbuf of the tap interface with vhost-net To: Harold Huang Cc: users@dpdk.org, Maxime Coquelin , Chenbo Xia , netdev Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Adding netdev. On Wed, Feb 23, 2022 at 9:46 PM Harold Huang wrote: > > Sorry. The performance tested by iperf is degraded from 4.5 Gbps to > 750Mbps per flow. > > Harold Huang =E4=BA=8E2022=E5=B9=B42=E6=9C=8823= =E6=97=A5=E5=91=A8=E4=B8=89 21:13=E5=86=99=E9=81=93=EF=BC=9A > > > > I see in dpdk virtio-user driver, the TUNSETSNDBUF is initialized with > > INT_MAX, see: https://github.com/DPDK/dpdk/blob/main/drivers/net/virtio= /virtio_user/vhost_kernel_tap.c#L169 Note that Linux use INT_MAX as default sndbuf for tuntap. > > It is ok because tap driver uses it to support tx baching, see this > > patch: https://github.com/torvalds/linux/commit/0a0be13b8fe2cac11da2063= fb03f0f39359b3069 > > > > But in tun_xdp_one, napi is not supported and I want to user napi in > > tun_get_user to enable gro. NAPI is not enabled in this path, want to send a patch to do that? Btw, NAPI mode is used for kernel networking stack hardening at start, but it would be interesting to see if it helps for the performance. > > As I result, I change the sndbuf to a > > value such as 212992 in /proc/sys/net/core/wmem_default. Can you describe your setup in detail? Where did you run the iperf server and client and where did you change the wmem_default? > > But the > > performance tested by iperf is greatly degraded, from 4.5 Gbps to > > 750Gbps per flow. I see the the iperf server consume 100% cpu core, > > which should be the bottleneck of the this test. The perf top result > > of iperf server cpu core is as follows: > > > > ''' > > Samples: 72 of event 'cycles', 4000 Hz, Event count (approx.): > > 22685278 lost: 0/0 drop: 0/0 > > Overhead Shared O Symbol > > 59.86% [kernel] [k] report_bug > > 20.66% [kernel] [k] module_find_bug > > 6.51% [kernel] [k] common_interrupt > > 2.82% [kernel] [k] __slab_free > > 1.48% [kernel] [k] copy_user_enhanced_fast_string > > 1.44% [kernel] [k] __skb_datagram_iter > > 1.42% [kernel] [k] notifier_call_chain > > 1.41% [kernel] [k] irq_work_run_list > > 1.41% [kernel] [k] update_irq_load_avg > > 1.41% [kernel] [k] task_tick_fair > > 1.41% [kernel] [k] cmp_ex_search > > 0.16% [kernel] [k] __ghes_peek_estatus.isra.12 > > 0.02% [kernel] [k] acpi_os_read_memory > > 0.00% [kernel] [k] native_apic_mem_write > > ''' > > I am not clear about the test result. Can we change the sndbuf size in > > dpdk? Is any way to enable vhost_net to use napi without changing the > > tun kernel driver? You can do this by not using INT_MAX as sndbuf. Thanks >