From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by dpdk.org (Postfix) with ESMTP id AE1191B346 for ; Thu, 2 Nov 2017 04:41:57 +0100 (CET) Received: by mail-qk0-f177.google.com with SMTP id q83so5178774qke.6 for ; Wed, 01 Nov 2017 20:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=7Sd7eMMslWbuDUn1/90AClxXQI738ypxFsYBiX0e034=; b=an2Mrvip9Y1PH5xokPraQzX3h+2L2jAw+tSYurxAb9z93jzcThVG/evVy8lKLsQXb1 8AdAkRypcLZlYHtj0aOKSPIgEpgzyQL6ogtq5wiTaqiqmz1x0t3XCrHuRoZUsf4wAYMW d96ACa8E3bosW+yEKv93xlcL0IjLNt57i3kg5LX5F+t1cqs2bqU/s26yop9XQ6zQ6eMZ 0FjTJyFNjiZ1RkaKd4OdaNqwUph8QSa6vVmWRt1lV44or+HOboV6zq2IC39dAJvhbo9S kNKPngBMSM4dN3qReF88dIguTe+j3N1oHKk+rcPFV5fl8KR7h251XmAJlw60gcmWI50p XEZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=7Sd7eMMslWbuDUn1/90AClxXQI738ypxFsYBiX0e034=; b=SwTy7vvLtazY+Az7PZaRPx+XIlv3vPKxA2eOrEdGCDqteyv3CAIZDdvgSY9KXQxb3+ dDt+w6a0MMEJiXA89U7TnBKYMKdalSrwWPy3M4N37hVJt+AhxcXnlAkBeGuRc8uKByjE wM6iLjzjqyRZOJIsjOm2NqqduzdsK5D3BDL40M04kHb3DDp9q6iahv6kbCkAJy3iIg1t C6InbQathwFahPWgbEOwUJBlh50N9TuYeqaRdcYKN0vrCOD8h3j3PDDucptVjN1USHNu rawwwc36bk+CqNbc0oKqFxn7D7mgxQPDrQwPehaqew3JxZpKsRRLIRqerra2tCm5tQ5E Fupg== X-Gm-Message-State: AMCzsaUu1TsjRIGef52GTUpcZU774WpSeKxqfnjZXZbqCx5arDSFR9Vo kSxgUrv/Qk6xfzykquUolHgu/aKFgWXGCKGX3wM= X-Google-Smtp-Source: ABhQp+Ruy8asQQxgL+B+D54bMMeihl8WX7rD71w2L3uwietaDuQM3YPCZfufhaOYY0cHMry4NECxxGzkOiFotLP2UFI= X-Received: by 10.55.21.99 with SMTP id f96mr3058275qkh.315.1509594116947; Wed, 01 Nov 2017 20:41:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.98.181 with HTTP; Wed, 1 Nov 2017 20:41:56 -0700 (PDT) From: Sam Date: Thu, 2 Nov 2017 11:41:56 +0800 Message-ID: To: ovs-dev@openvswitch.org, dev@dpdk.org, QEMU Developers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Please help me to debug memory mis-unmap in ovs-dpdk. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Nov 2017 03:41:57 -0000 Hi all, I'm debugging a mis-unmap bug in ovs-dpdk, process is this: 1. start ovs-dpdk with 10G socket memory, then use `top`, ovs-vswitchd take 10G memory. command is bellow. 2. start qemu vm1 with vhost device as server to connect port on ovs-dpdk, command is bellow. qemu share 40G memory with ovs-dpdk, then use `top`, ovs-vswitchd take 50G memory. 3. start qemu vm2 with vhost device as server to connect port on ovs-dpdk, command is bellow. qemu share 20G memory with ovs-dpdk, then use `top`, ovs-vswitchd take 70G memory. 4. stop vm1, then use `top`, ovs-vswitchd take 50G memory (I think it should be 30G?). use `cat /proc/meminfo`, 70G memory is not released. Please tell me where is the code ovs-dpdk and qemu communicate with each other about the map/unmap memory region. Thank you~ Is it old bug? Refer this link: http://dev.dpdk.narkive.com/m04py1jJ/dpdk-dev-vhost-user-technical-isssues Command to start ovs-dpdk: ovs-vswitchd --dpdk -c 0x14 -n 4 --socket-mem 10240 --proc-type secondary -w 0000:01:00.0 -w 0000:01:00.1 -- unix:/usr/local/var/run/openvswitch/db.sock --pidfile --detach --log-file --mlockall --no-chdir --log-file=/usr/local/var/log/openvswitch/ovs-vswitchd.log --pidfile=/usr/local/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor Command to start qemu: /usr/local/bin/qemu-system-x86_64 -enable-kvm -cpu qemu64,+vmx,+ssse3,+sse4.1,+sse4.2,+x2apic,+aes,+avx,+vme,+pat,+ss,+pclmulqdq,+xsave,level=13 -machine pc,accel=kvm -chardev socket,id=hmqmondev,port=55919,host=127.0.0.1,nodelay,server,nowait -mon chardev=hmqmondev,id=hmqmon,mode=readline -rtc base=utc,clock=host,driftfix=none -usb -device usb-tablet -daemonize -nodefaults -nodefconfig -no-kvm-pit-reinjection -global kvm-pit.lost_tick_policy=discard -vga std -k en-us -smp 1 -name gangyewei-qemutime-07-1 -m 20480 -boot order=cdn -vnc :19,password -drive file=/opt/cloud/workspace/disks/4a5148da-e34e-4c81-aada-12a247c0337e,if=none,id=drive_0,format=qcow2,cache=none,aio=native -device virtio-blk-pci,id=dev_drive_0,drive=drive_0,bus=pci.0,addr=0x5 -drive file=/opt/cloud/workspace/disks/08eb3f72-07ba-44e9-8ee9-4ab662dbbd1f,if=none,id=drive_1,format=qcow2,cache=none,aio=native -device virtio-blk-pci,id=dev_drive_1,drive=drive_1,bus=pci.0,addr=0x6 -device ide-cd,drive=ide0-cd0,bus=ide.1,unit=1 -drive id=ide0-cd0,media=cdrom,if=none -chardev socket,id=char-n-f879ac2f,path=/usr/local/var/run/openvswitch/n-f879ac2f,server -netdev type=vhost-user,id=n-f879ac2f,chardev=char-n-f879ac2f,vhostforce=on -device virtio-net-pci,netdev=n-f879ac2f,mac=00:22:f8:79:ac:2f,id=netdev-n-f879ac2f,addr=0xf,speed=1000 -object memory-backend-file,id=mem,size=20480M,mem-path=/dev/hugepages,share=on -mem-prealloc -numa node,memdev=mem -pidfile /opt/cloud/workspace/servers/63ef2ec4-556f-47ea-93e0-23089bb59be5/pid -chardev socket,path=/opt/cloud/workspace/servers/63ef2ec4-556f-47ea-93e0-23089bb59be5/qga.sock,server,nowait,id=qga0 -device virtio-serial -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 -qmp unix:/opt/cloud/workspace/servers/63ef2ec4-556f-47ea-93e0-23089bb59be5/qmp.sock,server,nowait Process: [gangyewei@yf-mos-test-net07 63ef2ec4-556f-47ea-93e0-23089bb59be5]$ top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 38470 root 10 -10 79.751g 223160 10636 S 200.0 0.2 58:26.77 ovs-vswitchd 4575 root 20 0 41.338g 90036 2924 S 0.0 0.1 0:21.71 qemu-system-x86 5914 root 20 0 20.652g 66824 2876 S 0.0 0.1 0:11.90 qemu-system-x86 [gangyewei@yf-mos-test-net07 63ef2ec4-556f-47ea-93e0-23089bb59be5]$ cd ../46c035e2-605c-4997-b4df-15d13f97758f [gangyewei@yf-mos-test-net07 46c035e2-605c-4997-b4df-15d13f97758f]$ sudo sh stopvm Remove PID /opt/cloud/workspace/servers/46c035e2-605c-4997-b4df-15d13f97758f/pid Remove VNC /opt/cloud/workspace/servers/46c035e2-605c-4997-b4df-15d13f97758f/vnc [gangyewei@yf-mos-test-net07 46c035e2-605c-4997-b4df-15d13f97758f]$ top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 38470 root 10 -10 55.751g 223160 10636 S 200.3 0.2 59:30.31 ovs-vswitchd 5914 root 20 0 20.652g 66824 2876 S 0.0 0.1 0:11.92 qemu-system-x86 [gangyewei@yf-mos-test-net07 46c035e2-605c-4997-b4df-15d13f97758f]$ cat /proc/meminfo HugePages_Total: 112 HugePages_Free: 42