From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw0-f169.google.com (mail-yw0-f169.google.com [209.85.161.169]) by dpdk.org (Postfix) with ESMTP id 080C42C6B for ; Mon, 7 Mar 2016 16:31:01 +0100 (CET) Received: by mail-yw0-f169.google.com with SMTP id d65so26400644ywb.0 for ; Mon, 07 Mar 2016 07:31:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=rTDfyNu3HLwPqF/bQwbIkbSesEB22PFsMILuOC/1o98=; b=G3IoLHJUJyWjvdQncesgUsEFddLgSCauTF/EERVXn19dq40WjpPMLgCe8PLypGg40P w3+BSsrFxvPhkvahQW4BpACsIOVDgIFxQimCvtmqN3hrbhqHt6EZMCFyfxqUYXK2HRcH RgxS2RyFx4zoXW6WC+9Utpom4sei49UsEmFEj/ij5X+bEXFZy+KehqRX3LjUwC7PwFSj NQ3XRAHXDkquQUgpZRy7/6cAH/XjqmFWLVVwnFivTdTofO3PyrzcwONSmt66qAcY1Olt kwuQxUNwdYgVcFF3tv5kdccFGNfKwSNNMUQV6PPr84vcMv24xOqzhlLCe7Z5Q9TL30V6 U8MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=rTDfyNu3HLwPqF/bQwbIkbSesEB22PFsMILuOC/1o98=; b=Kp1HMgeINC1oGw/U6qvG6aE3nhDBehxm7R2oUlgz5CTIOHPwQLGqfCa9S46N/Ktp7c TGt7DN4TwtCtx7kYq7fpBqntdhWicDulld+jeef992ace9w1A9cFIFl28UkGJV7HL4Z4 V7i5x2Ds6VILBDKB1KNml1QXbRzt2qwEEV/MaqzD6tdUkf9wSnPq7qTdRZIH5hfqoLe6 wnHGH3xkDym9kp3mmb2HmCg/bRAMdIxfP38ZVIVAVKBbhy/wB3o0xWHQRf0EeQS2d+Gw 1EabgiNZ9uaQEo0mwf8nmotSWncqYqNUyU/gJBiMhr/XrZwi5BOzzfya/dQ8V3ZDdcrS bL0g== X-Gm-Message-State: AD7BkJIMVr3rc75LEt9brWzBwdsyhqZWG5pN8+9eJm+fpI9tJgUAeK4csa28xapZucAUPA== X-Received: by 10.129.125.10 with SMTP id y10mr12640003ywc.331.1457364660507; Mon, 07 Mar 2016 07:31:00 -0800 (PST) Received: from mail-yk0-f182.google.com (mail-yk0-f182.google.com. [209.85.160.182]) by smtp.gmail.com with ESMTPSA id l128sm12230881ywd.5.2016.03.07.07.30.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Mar 2016 07:30:59 -0800 (PST) Received: by mail-yk0-f182.google.com with SMTP id 1so49614172ykg.3 for ; Mon, 07 Mar 2016 07:30:59 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.37.31.11 with SMTP id f11mr13173341ybf.110.1457364659447; Mon, 07 Mar 2016 07:30:59 -0800 (PST) Received: by 10.37.122.197 with HTTP; Mon, 7 Mar 2016 07:30:59 -0800 (PST) In-Reply-To: <116CFAF2-CD73-41C4-96C5-D748E90205F7@intel.com> References: <116CFAF2-CD73-41C4-96C5-D748E90205F7@intel.com> Date: Mon, 7 Mar 2016 23:30:59 +0800 X-Gmail-Original-Message-ID: Message-ID: From: Royce Niu To: "Wiles, Keith" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "users@dpdk.org" Subject: Re: [dpdk-users] Performance Problem of DPDK pkt-gen X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 15:31:01 -0000 Hi, Keith Maybe, since I didn't configure the CPU affinity in the L2FWD pc on purpose so far. But, my question is the first PC have a poor sending rate when L2FWD is working in second PC. You mean the problem is related to L2FWD? On Mon, Mar 7, 2016 at 11:26 PM, Wiles, Keith wrote: > >Dear all, > > > >I am using an server with 4 cpus (4 x 8 core CPUs with HT) and NICs > (X520). > > > >When I use pkt-gen on NIC1 or NIC2, the speed of generating 64Byte is > >14Mpps. > > > >If I generating both on NIC1 and NIC2, the speed of generating 64Byte on > >both are more than 13Mpps. > > > >However, I use same configuration PC (with DPDK L2FWD) to bridge NIC1 and > >NIC2, so I can generate packet on NIC1 and receive these packets on NIC2 > in > >pkt-gen. The speed of generating is decreased to 4Mpps and the receive > rate > >is 3Mpps. > > I am not sure how you configured the second PC for L2FWD, but I suspect > the L2FWD is having to receive packets on socket 0 and send the packets on > socket 1, this means the QPI bus gets involved here. Is this the case? > > > > > > >I want to know why generating speed is slower than the situation without > >the bridge of NIC1 and NIC2? how to solve these problem? > > > >The detailed information is as following. > > > >sudo sysctl vm.nr_hugepages=4096 > >echo 1024 | sudo tee > >/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages > >echo 1024 | sudo tee > >/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages > >echo 1024 | sudo tee > >/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages > >echo 1024 | sudo tee > >/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages > > > > > >sudo mkdir -p /dev/hugepages > >sudo mount -t hugetlbfs nodev /dev/hugepages > > > > > >sudo dpdk-2.2.0/tools/dpdk_nic_bind.py --status > >sudo modprobe uio > >sudo insmod dpdk-2.2.0/build/kmod/igb_uio.ko > > > >sudo dpdk-2.2.0/tools/dpdk_nic_bind.py -b igb_uio 04:00.0 04:00.1 > >sudo dpdk-2.2.0/tools/dpdk_nic_bind.py --status > > > >cd pktgen-2.9.12/ > > > >sudo app/build/pktgen -c 0x1f -n 3 --proc-type auto --socket-mem > >128,128,128,128 -- -P -m "[1:3].0, [2:4].1" -f test/set_seq.pkt > > > >I tried to change -m, but, sometime there is no packet generated by > >pkt-gen. > > > > > >The core map is : > > > >EAL: Detected lcore 0 as core 0 on socket 0 > >EAL: Detected lcore 1 as core 0 on socket 1 > >EAL: Detected lcore 2 as core 0 on socket 2 > >EAL: Detected lcore 3 as core 0 on socket 3 > >EAL: Detected lcore 4 as core 1 on socket 0 > >EAL: Detected lcore 5 as core 1 on socket 1 > >EAL: Detected lcore 6 as core 1 on socket 2 > >EAL: Detected lcore 7 as core 1 on socket 3 > >EAL: Detected lcore 8 as core 2 on socket 0 > >EAL: Detected lcore 9 as core 2 on socket 1 > >EAL: Detected lcore 10 as core 2 on socket 2 > >EAL: Detected lcore 11 as core 2 on socket 3 > >EAL: Detected lcore 12 as core 3 on socket 0 > >EAL: Detected lcore 13 as core 3 on socket 1 > >EAL: Detected lcore 14 as core 3 on socket 2 > >EAL: Detected lcore 15 as core 3 on socket 3 > >EAL: Detected lcore 16 as core 4 on socket 0 > >EAL: Detected lcore 17 as core 4 on socket 1 > >EAL: Detected lcore 18 as core 4 on socket 2 > >EAL: Detected lcore 19 as core 4 on socket 3 > >EAL: Detected lcore 20 as core 5 on socket 0 > >EAL: Detected lcore 21 as core 5 on socket 1 > >EAL: Detected lcore 22 as core 5 on socket 2 > >EAL: Detected lcore 23 as core 5 on socket 3 > >EAL: Detected lcore 24 as core 6 on socket 0 > >EAL: Detected lcore 25 as core 6 on socket 1 > >EAL: Detected lcore 26 as core 6 on socket 2 > >EAL: Detected lcore 27 as core 6 on socket 3 > >EAL: Detected lcore 28 as core 7 on socket 0 > >EAL: Detected lcore 29 as core 7 on socket 1 > >EAL: Detected lcore 30 as core 7 on socket 2 > >EAL: Detected lcore 31 as core 7 on socket 3 > >EAL: Detected lcore 32 as core 0 on socket 0 > >EAL: Detected lcore 33 as core 0 on socket 1 > >EAL: Detected lcore 34 as core 0 on socket 2 > >EAL: Detected lcore 35 as core 0 on socket 3 > >EAL: Detected lcore 36 as core 1 on socket 0 > >EAL: Detected lcore 37 as core 1 on socket 1 > >EAL: Detected lcore 38 as core 1 on socket 2 > >EAL: Detected lcore 39 as core 1 on socket 3 > >EAL: Detected lcore 40 as core 2 on socket 0 > >EAL: Detected lcore 41 as core 2 on socket 1 > >EAL: Detected lcore 42 as core 2 on socket 2 > >EAL: Detected lcore 43 as core 2 on socket 3 > >EAL: Detected lcore 44 as core 3 on socket 0 > >EAL: Detected lcore 45 as core 3 on socket 1 > >EAL: Detected lcore 46 as core 3 on socket 2 > >EAL: Detected lcore 47 as core 3 on socket 3 > >EAL: Detected lcore 48 as core 4 on socket 0 > >EAL: Detected lcore 49 as core 4 on socket 1 > >EAL: Detected lcore 50 as core 4 on socket 2 > >EAL: Detected lcore 51 as core 4 on socket 3 > >EAL: Detected lcore 52 as core 5 on socket 0 > >EAL: Detected lcore 53 as core 5 on socket 1 > >EAL: Detected lcore 54 as core 5 on socket 2 > >EAL: Detected lcore 55 as core 5 on socket 3 > >EAL: Detected lcore 56 as core 6 on socket 0 > >EAL: Detected lcore 57 as core 6 on socket 1 > >EAL: Detected lcore 58 as core 6 on socket 2 > >EAL: Detected lcore 59 as core 6 on socket 3 > >EAL: Detected lcore 60 as core 7 on socket 0 > >EAL: Detected lcore 61 as core 7 on socket 1 > >EAL: Detected lcore 62 as core 7 on socket 2 > >EAL: Detected lcore 63 as core 7 on socket 3 > > > > > > > > > > > > > > > > > >-- > >Regards, > > > >Royce Niu > > > > > Regards, > Keith > > > > > -- Regards, Royce Niu