From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by dpdk.org (Postfix) with ESMTP id 2D0E45A45 for ; Thu, 14 May 2015 16:16:11 +0200 (CEST) Received: by wicmx19 with SMTP id mx19so16191605wic.0 for ; Thu, 14 May 2015 07:16:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EiLwNfpdqGp0K+SUhP9Xxqr94q/SWoWLqqsd64eVscM=; b=LAfqTE/vP1LjDrd1AbIYs3fOTHa70cqrLeB5U+NHJX5lcQ1sc/GSsEDJcGjL4KfNcG x25K0PmuC2fpveh/UGdeV+Edk6ngGGa9/TzuDVo7T83rmqWqz/h+I+zFBHkkATepqBOF dCkJROtc4g/GOLqW6pkDnJ0dBKgn1egVaSihtC19HlmdetvRHvHjBg3gTB2sYLBg1Xr7 mBiy7hztALrq0HbDQk/mQGEXIs8Mi2VHY9sqCehntyUSrjzV0jFZsSEI36hRZgHt2G+h v0T+HwGWBbhVCIMY0SFhoSiBc845/fY+lOUI0+zg4c1ZvahBruN3zzQWFh/5fJcA6n5S rdYw== X-Gm-Message-State: ALoCoQmYFBs8xcAA1gWPqWBIjcjbta+0TBShUQWDHweHcQsbWLmfVNaYkPn73IHk5IopO8yywQVu MIME-Version: 1.0 X-Received: by 10.181.13.44 with SMTP id ev12mr24174610wid.17.1431612969944; Thu, 14 May 2015 07:16:09 -0700 (PDT) Received: by 10.27.127.86 with HTTP; Thu, 14 May 2015 07:16:09 -0700 (PDT) In-Reply-To: References: Date: Thu, 14 May 2015 16:16:09 +0200 Message-ID: From: Maciej Grochowski To: "Xie, Huawei" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Issues with example/vhost with running VM X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2015 14:16:11 -0000 Thank You Xie for reply, When I run vhost with -m 2048 it didn't start, with -m 3000 and host with 1024 give me segfault (I calculated hugepages and have only 3 free) VHOST_CONFIG: bind to usvhost VHOST_CONFIG: new virtio connection is 26 VHOST_CONFIG: new device, handle is 0 VHOST_CONFIG: new virtio connection is 27 VHOST_CONFIG: new device, handle is 1 VHOST_CONFIG: read message VHOST_USER_SET_OWNER VHOST_CONFIG: read message VHOST_USER_GET_FEATURES VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:0 file:28 VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:1 file:29 VHOST_CONFIG: read message VHOST_USER_SET_OWNER VHOST_CONFIG: read message VHOST_USER_GET_FEATURES VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:0 file:30 VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:1 file:31 VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:0 file:32 VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:1 file:28 VHOST_CONFIG: read message VHOST_USER_SET_FEATURES VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE VHOST_CONFIG: mapped region 0 fd:29 to 0x2aaaaac00000 sz:0xa0000 off:0x0 VHOST_CONFIG: mapped region 1 fd:33 to 0xffffffffffffffff sz:0x50000000 off:0xc0000 VHOST_CONFIG: mmap qemu guest failed. VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR Segmentation fault (core dumped) While testing different configuration I finally got -m 3584 ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge --socket-mem 3712 -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9 so I can run vhost-switch from 3584*2048k Hugepage -with this vhost user I run two KVM machines with followed parameters kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 -cpu host -smp 2 -hda /home/ubuntu/qemu/debian_squeeze2_amd64.qcow2 -m 1024 -mem-path /mnt/huge -mem-prealloc -chardev socket,id=char1,path=/home/ubuntu/dpdk/examples/vhost/usvhost -netdev type=vhost-user,id=hostnet1,chardev=char1 -device virtio-net pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off -chardev socket,id=char2,path=/home/ubuntu/dpdk/examples/vhost/usvhost -netdev type=vhost-user,id=hostnet2,chardev=char2 -device virtio-net-pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off with this i got ... VHOST_CONFIG: mapped region 0 fd:31 to 0x2aaabae00000 sz:0xa0000 off:0x0 VHOST_CONFIG: mapped region 1 fd:37 to 0x2aaabb000000 sz:0x10000000 off:0xc0000 VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK VHOST_CONFIG: vring kick idx:0 file:38 VHOST_CONFIG: virtio isn't ready for processing. VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK VHOST_CONFIG: vring kick idx:1 file:39 VHOST_CONFIG: virtio is now ready for processing. VHOST_DATA: (1) Device has been added to data core 2 So everything looking ok, Thank You Xie. But I found another Issue, maybe it is something trivial but using options: --vm2vm 1 (or) 2 --stats 9 it seems that I didn't have connection between VM2VM communication. I set manually IP for eth0 and eth1: on 1 VM ifconfig eth0 192.168.0.100 netmask 255.255.255.0 up ifconfig eth1 192.168.1.101 netmask 255.255.255.0 up on 2 VM ifconfig eth0 192.168.1.200 netmask 255.255.255.0 up ifconfig eth1 192.168.0.202 netmask 255.255.255.0 up And I cant ping between any IP/interfaces, moreover stats show me that: Device statistics ==================================== Statistics for device 0 ------------------------------ TX total: 0 TX dropped: 0 TX successful: 0 RX total: 0 RX dropped: 0 RX successful: 0 Statistics for device 1 ------------------------------ TX total: 0 TX dropped: 0 TX successful: 0 RX total: 0 RX dropped: 0 RX successful: 0 Statistics for device 2 ------------------------------ TX total: 0 TX dropped: 0 TX successful: 0 RX total: 0 RX dropped: 0 RX successful: 0 Statistics for device 3 ------------------------------ TX total: 0 TX dropped: 0 TX successful: 0 RX total: 0 RX dropped: 0 RX successful: 0 ====================================================== So it seems like any packet didn't leave my VM. also arp table is empty on each VM Do You have any ide what can be wrong with configuration between VM2VM? On Wed, May 13, 2015 at 7:53 PM, Xie, Huawei wrote: > Try --socket-mem or -m 2048 to limit the vhost switch's memory > consumption, note that vswitch requires several GB memory due to some issue > in the example, so try allocating more huges pages. > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Maciej Grochowski > > Sent: Wednesday, May 13, 2015 11:00 PM > > To: dev@dpdk.org > > Subject: [dpdk-dev] Issues with example/vhost with running VM > > > > Hello, I trying to create vm2vm benchmark on my ubuntu(14.04) based > > platform. > > > > I had compiled DPDK to run a vhost example app with followed flags > > > > CONFIG_RTE_LIBRTE_VHOST=y > > CONFIG_RTE_LIBRTE_VHOST_USER=y > > CONFIG_RTE_LIBRTE_VHOST_DEBUG=n > > > > > > then I run vhost app based on documentation: > > > > ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge -- -p 0x1 > > --dev-basename usvhost > > > > then I trying to start kvm VM > > > > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 -cpu > > host > > -smp 2 -mem-path /mnt/huge -mem-prealloc \ > > -hda /home/ubuntu/qemu/debian_squeeze2_amd64.qcow2 -m 4096 \ > > -chardev > > socket,id=char1,path=/home/ubuntu/dpdk/examples/vhost/usvhost \ > > -netdev type=vhost-user,id=hostnet1,chardev=char1 \ > > -device > > virtio-net- > > pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=o > > ff,guest_ecn=off > > \ > > -chardev > > socket,id=char2,path=/home/ubuntu/dpdk/examples/vhost/usvhost \ > > -netdev type=vhost-user,id=hostnet2,chardev=char2 \ > > -device > > virtio-net- > > pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6=o > > ff,guest_ecn=off > > > > but this give me an error: > > > > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet1,chardev=char1: > > chardev "char1" went up > > qemu-system-x86_64: unable to map backing store for hugepages: Cannot > > allocate memory > > > > > > On vhost app in logs I can see: > > > > VHOST_DATA: Procesing on Core 1 started > > VHOST_DATA: Procesing on Core 2 started > > VHOST_DATA: Procesing on Core 3 started > > VHOST_CONFIG: socket created, fd:25 > > VHOST_CONFIG: bind to usvhost > > VHOST_CONFIG: new virtio connection is 26 > > VHOST_CONFIG: new device, handle is 0 > > VHOST_CONFIG: read message VHOST_USER_SET_OWNER > > VHOST_CONFIG: read message VHOST_USER_GET_FEATURES > > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL > > VHOST_CONFIG: vring call idx:0 file:27 > > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL > > VHOST_CONFIG: vring call idx:1 file:28 > > VHOST_CONFIG: recvmsg failed > > VHOST_CONFIG: vhost peer closed > > > > > > So that looks at huge page memory problem. > > On my machine I had 2048k huge pages, and I can allocate 2479. > > > > before I run vhost "cat /proc/meminfo | grep Huge" show > > > > AnonHugePages: 4096 kB > > HugePages_Total: 2479 > > HugePages_Free: 2479 > > HugePages_Rsvd: 0 > > HugePages_Surp: 0 > > Hugepagesize: 2048 kB > > > > and while running vhost: > > > > AnonHugePages: 4096 kB > > HugePages_Total: 2479 > > HugePages_Free: 0 > > HugePages_Rsvd: 0 > > HugePages_Surp: 0 > > Hugepagesize: 2048 kB > > > > so that looks that I didn't have free hugepages for my VM. But this looks > > as independently if I reserve 1k 2k or 2.5k memory always example/vhost > got > > whole memory. > > > > Any help will be greatly appreciated >