From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb0-f174.google.com (mail-yb0-f174.google.com [209.85.213.174]) by dpdk.org (Postfix) with ESMTP id C8F4C5681 for ; Thu, 22 Sep 2016 16:41:38 +0200 (CEST) Received: by mail-yb0-f174.google.com with SMTP id 2so49118279ybv.0 for ; Thu, 22 Sep 2016 07:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7dIQ+QOIHuYdUwTZ/Tq4SMnwdlHnCzV7WByKsgxaKC4=; b=jTUK+DVXYdPqzoAOq3dB+7L805UNYL99vT7OUmuzCdlOPGQt4Qph0dCjQPcG2EJ3pZ rI+aRvdqZpuFulPiKviq5XJZgCEef16aHW1KQbiNi8O81DbWk+HppzAmS9j7nJDAgwUR MgLCyhB+INvTYSztCQHq5kTz71AfV0AFXYNHs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7dIQ+QOIHuYdUwTZ/Tq4SMnwdlHnCzV7WByKsgxaKC4=; b=M5xtBMeqgE+L0wWrneAuuOuympd1d9sbGMzy3QPuxBGYAeOWpL+xiPOkbe/AvOmtDJ 6NINBQ2QXtKzGbuAERr4DUchcoPKcHQ5o+brS8NJJOUB9A/i7nmTQHqK3Rz5JxkSMcWu v5NfbCH9WNJJVoXMcQFh2LZcc890WBTadsWDDgca9p5rsg7ONeFMUhhgqMsMlGmlKAV6 ma8oXqVHrqdzWovpCw6JddBCTlQpltl2f8+L0lXBdGvYsRxQPJKh8mM9wS5k4g3Bp6fU W05WrYZIb9psFc2KZwVWDgAUigc5LE9PVC4274FbB0vwYnukc1cxNo0W2pqaM5xAHJeA AKSA== X-Gm-Message-State: AE9vXwMtY8QbpE0wkZZEjD2ILNm0zuflLYkdYnB3vr/FZhEAU0qNSzS94K1RPYTFkK3KRZICjZMzlvK0lI0g4Rjf X-Received: by 10.37.33.197 with SMTP id h188mr1747163ybh.82.1474555298193; Thu, 22 Sep 2016 07:41:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.25.6 with HTTP; Thu, 22 Sep 2016 07:41:37 -0700 (PDT) In-Reply-To: <8F6C2BD409508844A0EFC19955BE09414E7B5ED5@SHSMSX103.ccr.corp.intel.com> References: <1471319402-112998-1-git-send-email-zhihong.wang@intel.com> <1471585430-125925-1-git-send-email-zhihong.wang@intel.com> <8F6C2BD409508844A0EFC19955BE09414E7B5581@SHSMSX103.ccr.corp.intel.com> <20160922022903.GJ23158@yliu-dev.sh.intel.com> <8F6C2BD409508844A0EFC19955BE09414E7B5DAE@SHSMSX103.ccr.corp.intel.com> <8F6C2BD409508844A0EFC19955BE09414E7B5ED5@SHSMSX103.ccr.corp.intel.com> From: Jianbo Liu Date: Thu, 22 Sep 2016 22:41:37 +0800 Message-ID: To: "Wang, Zhihong" Cc: Yuanhan Liu , Maxime Coquelin , "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 Subject: Re: [dpdk-dev] [PATCH v3 0/5] vhost: optimize enqueue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 14:41:39 -0000 On 22 September 2016 at 18:04, Wang, Zhihong wrote: > > >> -----Original Message----- >> From: Jianbo Liu [mailto:jianbo.liu@linaro.org] >> Sent: Thursday, September 22, 2016 5:02 PM >> To: Wang, Zhihong >> Cc: Yuanhan Liu ; Maxime Coquelin >> ; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH v3 0/5] vhost: optimize enqueue >> >> On 22 September 2016 at 14:58, Wang, Zhihong >> wrote: >> > >> > >> >> -----Original Message----- >> >> From: Jianbo Liu [mailto:jianbo.liu@linaro.org] >> >> Sent: Thursday, September 22, 2016 1:48 PM >> >> To: Yuanhan Liu >> >> Cc: Wang, Zhihong ; Maxime Coquelin >> >> ; dev@dpdk.org >> >> Subject: Re: [dpdk-dev] [PATCH v3 0/5] vhost: optimize enqueue >> >> >> >> On 22 September 2016 at 10:29, Yuanhan Liu >> >> >> wrote: >> >> > On Wed, Sep 21, 2016 at 08:54:11PM +0800, Jianbo Liu wrote: >> >> >> >> > My setup consists of one host running a guest. >> >> >> >> > The guest generates as much 64bytes packets as possible using >> >> >> >> >> >> >> >> Have you tested with other different packet size? >> >> >> >> My testing shows that performance is dropping when packet size is >> >> more >> >> >> >> than 256. >> >> >> > >> >> >> > >> >> >> > Hi Jianbo, >> >> >> > >> >> >> > Thanks for reporting this. >> >> >> > >> >> >> > 1. Are you running the vector frontend with mrg_rxbuf=off? >> >> >> > >> >> Yes, my testing is mrg_rxbuf=off, but not vector frontend PMD. >> >> >> >> >> > 2. Could you please specify what CPU you're running? Is it Haswell >> >> >> > or Ivy Bridge? >> >> >> > >> >> It's an ARM server. >> >> >> >> >> > 3. How many percentage of drop are you seeing? >> >> The testing result: >> >> size (bytes) improvement (%) >> >> 64 3.92 >> >> 128 11.51 >> >> 256 24.16 >> >> 512 -13.79 >> >> 1024 -22.51 >> >> 1500 -12.22 >> >> A correction is that performance is dropping if byte size is larger than 512. >> > >> > >> > Jianbo, >> > >> > Could you please verify does this patch really cause enqueue perf to drop? >> > >> > You can test the enqueue path only by set guest to do rxonly, and compare >> > the mpps by show port stats all in the guest. >> > >> > >> Tested with testpmd, host: txonly, guest: rxonly >> size (bytes) improvement (%) >> 64 4.12 >> 128 6 >> 256 2.65 >> 512 -1.12 >> 1024 -7.02 > > > > I think your number is little bit hard to understand for me, this patch's > optimization contains 2 parts: > > 1. ring operation: works for both mrg_rxbuf on and off > > 2. remote write ordering: works for mrg_rxbuf=on only > > So, for mrg_rxbuf=off, if this patch is good for 64B packets, then it > shouldn't do anything bad for larger packets. > > This is the gain on x86 platform: host iofwd between nic and vhost, > guest rxonly. > > nic2vm enhancement > 64 21.83% > 128 16.97% > 256 6.34% > 512 0.01% > 1024 0.00% > I bootup a VM with 2 virtual port, and stress the traffic between them. First, I stressed with pktgen-dpdk in VM, and did iofwd in host. Then, as you told, I did rxonly in VM, and txonly in host. > I suspect there's some complication in ARM's micro-arch. > > Could you try v6 and apply all patches except the the last one: > [PATCH v6 6/6] vhost: optimize cache access > > And see if there's still perf drop? > The last patch can improve the performance. The drop is actually caused by the second patch. Jianbo