Re: [dpdk-dev] ovs crash when running traffic from VM to VM over DPDK and vhostuser

[Yi Ba <yby.developer@yahoo.com>]

Running with dpdk 16.04 and latest ovs from git, and removing "mrg_rxbuf=off" from virtio params, the crash is no longer observed. However, we are wittnessing ovs gets stuck, and will post to ovs mailing list:2016-05-02T17:26:18.804Z|00111|ovs_rcu|WARN|blocked 1000 ms waiting for pmd145 to quiesce
2016-05-02T17:26:19.805Z|00112|ovs_rcu|WARN|blocked 2001 ms waiting for pmd145 to quiesce
2016-05-02T17:26:21.804Z|00113|ovs_rcu|WARN|blocked 4000 ms waiting for pmd145 to quiesce
2016-05-02T17:26:25.805Z|00114|ovs_rcu|WARN|blocked 8001 ms waiting for pmd145 to quiesce
2016-05-02T17:26:33.805Z|00115|ovs_rcu|WARN|blocked 16001 ms waiting for pmd145 to quiesce
2016-05-02T17:26:49.805Z|00116|ovs_rcu|WARN|blocked 32001 ms waiting for pmd145 to quiesce
2016-05-02T17:27:14.354Z|00072|ovs_rcu(vhost_thread2)|WARN|blocked 128000 ms waiting for pmd145 to quiesce
2016-05-02T17:27:15.841Z|00008|ovs_rcu(urcu3)|WARN|blocked 128001 ms waiting for pmd145 to quiesce
2016-05-02T17:27:21.805Z|00117|ovs_rcu|WARN|blocked 64000 ms waiting for pmd145 to quiesce
2016-05-02T17:28:25.804Z|00118|ovs_rcu|WARN|blocked 128000 ms waiting for pmd145 to quiesce
 

    On Wednesday, 6 April 2016 10:56 AM, Yuanhan Liu <yuanhan.liu@linux.intel.com> wrote:
 

 On Tue, Apr 05, 2016 at 08:36:19PM +0000, Yi Ba wrote:
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ff1ddffb700 (LWP 21287)]
> 0x0000000000450da7 in update_secure_len (vec_idx=0x7ff1ddff27f8, secure_len=0x7ff1ddff27fc, id=13948, vq=0x7fe7992c8940)
>    at /home/stack/ovs-dpdk/dpdk-2.2.0/lib/librte_vhost/vhost_rxtx.c:452
> 452    /home/stack/ovs-dpdk/dpdk-2.2.0/lib/librte_vhost/vhost_rxtx.c: No such file or directory.
> (gdb) bt
> #0  0x0000000000450da7 in update_secure_len (vec_idx=0x7ff1ddff27f8, secure_len=0x7ff1ddff27fc, id=13948, vq=0x7fe7992c8940)

It looks like a known issue, which has been fixed in this release. So,
could you please just try again with the latest DPDK code? It should
be able to solve your issue.

    --yliu



From yuanhan.liu@linux.intel.com  Mon May  2 23:20:28 2016
Return-Path: <yuanhan.liu@linux.intel.com>
Received: from mga04.intel.com (mga04.intel.com [192.55.52.120])
 by dpdk.org (Postfix) with ESMTP id 1A22639EA
 for <dev@dpdk.org>; Mon,  2 May 2016 23:20:27 +0200 (CEST)
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by fmsmga104.fm.intel.com with ESMTP; 02 May 2016 14:20:28 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.24,569,1455004800"; d="scan'208";a="95880071"
Received: from yliu-dev.sh.intel.com ([10.239.67.162])
 by fmsmga004.fm.intel.com with ESMTP; 02 May 2016 14:20:26 -0700
From: Yuanhan Liu <yuanhan.liu@linux.intel.com>
To: dev@dpdk.org
Cc: huawei.xie@intel.com,
	Yuanhan Liu <yuanhan.liu@linux.intel.com>
Date: Mon,  2 May 2016 14:23:42 -0700
Message-Id: <1462224230-19460-1-git-send-email-yuanhan.liu@linux.intel.com>
X-Mailer: git-send-email 1.9.0
In-Reply-To: <1461645951-14603-1-git-send-email-yuanhan.liu@linux.intel.com>
References: <1461645951-14603-1-git-send-email-yuanhan.liu@linux.intel.com>
Subject: [dpdk-dev] [PATCH v2 0/8] vhost/example cleanup/fix
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 02 May 2016 21:20:28 -0000

I'm starting to work on the vhost ABI refactoring, that I also have to
touch the vhost example code. The vhost example code, however, is very
messy, full of __very__ long lines. This would make a later diff to
apply the new vhost API be very ugly, therefore, not friendly for review.
This is how this cleanup comes.

Besides that, there is one enhancement patch, which handles the broadcast
packets so that we could rely the ARP request packet, to let vhost-switch
be more like a real switch. There is another patch that (hopefully) would
fix the mbuf allocation failure ultimately. I also added some guidelines
there as comments to show how to count how many mbuf entries is enough for
our usage.

In another word, an example is meant to be clean/simple and with good
coding style so that people can get the usage easily. So, one way or
another, this patch is good to have, even without this ABI refactoring
stuff.

Note that I'm going to apply it before the end of this week, if no objections.


v2: - some checkpatch fixes

    - cleaned the code about device statistics

---
Yuanhan Liu (8):
  examples/vhost: remove the non-working zero copy code
  examples/vhost: remove unused macro and struct
  examples/vhost: use tailq to link vhost devices
  examples/vhost: use mac compare helper function directly
  examples/vhost: handle broadcast packet
  examples/vhost: fix mbuf allocation failure
  examples/vhost: switch_worker cleanup
  examples/vhost: embed statistics into vhost_dev struct

 doc/guides/sample_app_ug/vhost.rst |   36 +-
 examples/vhost/main.c              | 2394 ++++++------------------------------
 examples/vhost/main.h              |   56 +-
 3 files changed, 391 insertions(+), 2095 deletions(-)

--
1.9.3