From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sami.assaad@alcatel-lucent.com>
Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com
 [135.245.210.21]) by dpdk.org (Postfix) with ESMTP id 40A46C494
 for <dev@dpdk.org>; Mon, 29 Jun 2015 16:02:48 +0200 (CEST)
Received: from us70uusmtp3.zam.alcatel-lucent.com (unknown [135.5.2.65])
 by Websense Email Security Gateway with ESMTPS id C1CBB4608041B;
 Mon, 29 Jun 2015 14:02:44 +0000 (GMT)
Received: from US70UWXCHHUB02.zam.alcatel-lucent.com
 (us70uwxchhub02.zam.alcatel-lucent.com [135.5.2.49])
 by us70uusmtp3.zam.alcatel-lucent.com (GMO) with ESMTP id t5TE2kp3030703
 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL);
 Mon, 29 Jun 2015 14:02:46 GMT
Received: from US70TWXCHMBA07.zam.alcatel-lucent.com ([169.254.1.180]) by
 US70UWXCHHUB02.zam.alcatel-lucent.com ([135.5.2.49]) with mapi id
 14.03.0195.001; Mon, 29 Jun 2015 10:02:46 -0400
From: "Assaad, Sami (Sami)" <sami.assaad@alcatel-lucent.com>
To: Bruce Richardson <bruce.richardson@intel.com>
Thread-Topic: [dpdk-dev] 10G Interface used as PCI Pass-Through reports
 64bytes / packet
Thread-Index: AdCwMQTUCXg/w0KeRdCfiJzhxdVx0wCPROcAAACzgQA=
Date: Mon, 29 Jun 2015 14:02:45 +0000
Message-ID: <9478F0FB69DAA249AF0A9BDA1E6ED952188433D5@US70TWXCHMBA07.zam.alcatel-lucent.com>
References: <9478F0FB69DAA249AF0A9BDA1E6ED95218841F3C@US70TWXCHMBA07.zam.alcatel-lucent.com>
 <20150629091832.GA10380@bricha3-MOBL3>
In-Reply-To: <20150629091832.GA10380@bricha3-MOBL3>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.5.27.17]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] 10G Interface used as PCI Pass-Through reports
 64bytes / packet
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 29 Jun 2015 14:02:48 -0000

Thanks Bruce for your response.

This is a very odd issue. I'm connecting a Pktgen-DPDK server directly to o=
ur application server running a DPDK Server/Client Process architecture. I'=
m constantly seeing 64bytes per packet, no matter what packet size is being=
 received by the NIC running on the application server. I initially thought=
 that maybe the NIC was configured in a particular mode to improve the thro=
ughput through the SR-IOV links. Based on your response, this is a wrong as=
sumption. Interesting...I have tried both a HP and Intel NIC, DPDK 1.8 and =
2.0, and DPDK examples, all of which report 64 bytes/packet by the NIC port=
s. I'm using CentOS 6.6 (2.6.32-504.23.4.el6.x86_64). I'm wondering if my P=
CI pass-through is setup properly (?) ... yet, all the network traffic is b=
eing handled as expected by the designed DPDK application.

In case the issue is with the VM configuration (which I seriously doubt), I=
've copied my VM XML here:
<domain type=3D'kvm'>
  <name>vm-sami</name>
  <uuid>1eda9ae3-0155-de14-6e1c-0fbe0aa880f6</uuid>
  <memory unit=3D'KiB'>102400000</memory>
  <currentMemory unit=3D'KiB'>102400000</currentMemory>
  <vcpu placement=3D'static'>46</vcpu>
  <os>
    <type arch=3D'x86_64' machine=3D'rhel6.6.0'>hvm</type>
    <boot dev=3D'hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode=3D'custom' match=3D'exact'>
    <model fallback=3D'forbid'>Haswell</model>
    <vendor>Intel</vendor>
    <feature policy=3D'disable' name=3D'rtm'/>
    <feature policy=3D'disable' name=3D'hle'/>
    <feature policy=3D'require' name=3D'vme'/>
    <feature policy=3D'require' name=3D'dtes64'/>
    <feature policy=3D'require' name=3D'invpcid'/>
    <feature policy=3D'require' name=3D'vmx'/>
    <feature policy=3D'require' name=3D'erms'/>
    <feature policy=3D'require' name=3D'xtpr'/>
    <feature policy=3D'require' name=3D'smep'/>
    <feature policy=3D'require' name=3D'pbe'/>
    <feature policy=3D'require' name=3D'est'/>
    <feature policy=3D'require' name=3D'monitor'/>
    <feature policy=3D'require' name=3D'smx'/>
    <feature policy=3D'require' name=3D'abm'/>
    <feature policy=3D'require' name=3D'tm'/>
    <feature policy=3D'require' name=3D'acpi'/>
    <feature policy=3D'require' name=3D'fma'/>
    <feature policy=3D'require' name=3D'osxsave'/>
    <feature policy=3D'require' name=3D'ht'/>
    <feature policy=3D'require' name=3D'dca'/>
    <feature policy=3D'require' name=3D'pdcm'/>
    <feature policy=3D'require' name=3D'pdpe1gb'/>
    <feature policy=3D'require' name=3D'fsgsbase'/>
    <feature policy=3D'require' name=3D'f16c'/>
    <feature policy=3D'require' name=3D'ds'/>
    <feature policy=3D'require' name=3D'invtsc'/>
    <feature policy=3D'require' name=3D'tm2'/>
    <feature policy=3D'require' name=3D'avx2'/>
    <feature policy=3D'require' name=3D'ss'/>
    <feature policy=3D'require' name=3D'bmi1'/>
    <feature policy=3D'require' name=3D'bmi2'/>
    <feature policy=3D'require' name=3D'pcid'/>
    <feature policy=3D'require' name=3D'ds_cpl'/>
    <feature policy=3D'require' name=3D'movbe'/>
    <feature policy=3D'require' name=3D'rdrand'/>
  </cpu>
  <clock offset=3D'utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type=3D'file' device=3D'disk'>
      <driver name=3D'qemu' type=3D'qcow2' cache=3D'none'/>
      <source file=3D'...'/>
      <target dev=3D'vda' bus=3D'virtio'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x06' fu=
nction=3D'0x0'/>
    </disk>
    <controller type=3D'usb' index=3D'0' model=3D'ich9-ehci1'>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x05' fu=
nction=3D'0x7'/>
    </controller>
    <controller type=3D'usb' index=3D'0' model=3D'ich9-uhci1'>
      <master startport=3D'0'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x05' fu=
nction=3D'0x0' multifunction=3D'on'/>
    </controller>
    <controller type=3D'usb' index=3D'0' model=3D'ich9-uhci2'>
      <master startport=3D'2'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x05' fu=
nction=3D'0x1'/>
    </controller>
    <controller type=3D'usb' index=3D'0' model=3D'ich9-uhci3'>
      <master startport=3D'4'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x05' fu=
nction=3D'0x2'/>
    </controller>
    <interface type=3D'bridge'>
      <mac address=3D'52:54:00:6d:39:c5'/>
      <source bridge=3D'br0'/>
      <model type=3D'virtio'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x03' fu=
nction=3D'0x0'/>
    </interface>
    <serial type=3D'pty'>
      <target port=3D'0'/>
    </serial>
    <console type=3D'pty'>
      <target type=3D'serial' port=3D'0'/>
    </console>
    <input type=3D'tablet' bus=3D'usb'/>
    <input type=3D'mouse' bus=3D'ps2'/>
    <graphics type=3D'vnc' port=3D'-1' autoport=3D'yes'/>
    <sound model=3D'ich6'>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x04' fu=
nction=3D'0x0'/>
    </sound>
    <video>
      <model type=3D'cirrus' vram=3D'9216' heads=3D'1'/>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x02' fu=
nction=3D'0x0'/>
    </video>
    <hostdev mode=3D'subsystem' type=3D'pci' managed=3D'yes'>
      <source>
        <address domain=3D'0x0000' bus=3D'0x04' slot=3D'0x00' function=3D'0=
x0'/>
      </source>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x07' fu=
nction=3D'0x0'/>
    </hostdev>
    <hostdev mode=3D'subsystem' type=3D'pci' managed=3D'yes'>
      <source>
        <address domain=3D'0x0000' bus=3D'0x04' slot=3D'0x00' function=3D'0=
x1'/>
      </source>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x08' fu=
nction=3D'0x0'/>
    </hostdev>
    <memballoon model=3D'virtio'>
      <address type=3D'pci' domain=3D'0x0000' bus=3D'0x00' slot=3D'0x09' fu=
nction=3D'0x0'/>
    </memballoon>
  </devices>
</domain>

=20
Best Regards,
Sami Assaad.

-----Original Message-----
From: Bruce Richardson [mailto:bruce.richardson@intel.com]=20
Sent: Monday, June 29, 2015 5:19 AM
To: Assaad, Sami (Sami)
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] 10G Interface used as PCI Pass-Through reports 64by=
tes / packet

On Fri, Jun 26, 2015 at 04:56:18PM +0000, Assaad, Sami (Sami) wrote:
> Hello,
>=20
> Is it normal that a 10G NIC interface, supporting the 82599 Ethernet Cont=
roller, configured as PCI Pass-through for a virtual machine using DPDK,  r=
eports 64 bytes per packet; no matter what the packet size?
>=20
That would not be expected behaviour, no.
AFAIK, the 82599 NIC counters should behave in the same way whether or not =
it is passed through to a VM or used on a host.=20

> If so; I'm assuming this is to improve the performance of passing the net=
work traffic to the VM.  Is there a way to configure the NIC to properly pr=
esent the proper byte count/packet?
>=20
I'm not sure what you mean here. I can't see how the reporting of byte-coun=
ts would affect performance. Can you clarify what exactly you are seeing, a=
nd why you think there is a performance benefit because of it?

/Bruce

> Thanks in advance.
>=20
> Best Regards,
> Sami.