[dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[Jan Kiszka]

Hi again,

two findings in the vhost-switch example code that can cause grey hair
for starters:

- MAX_QUEUES of 512 causes pretty high memory need for the application
  (something between 1 and 2G) - is that really needed? I'm now running
  with 32, and I'm able to get away with 256M. Can we tune this
  default?

- hw_strip_crc is set to 0, but either the igb driver or the ET2 quad
  port adapter I'm using is ignoring this. It does strip the CRC, so
  does software, and I'm losing 4 bytes on each unpadded packet. Known
  issue?

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[Ouyang, Changchun]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jan Kiszka
> Sent: Saturday, August 8, 2015 1:31 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] vhost-switch example: huge memory need and CRC off-
> loading issue
> 
> Hi again,
> 
> two findings in the vhost-switch example code that can cause grey hair for
> starters:
> 
> - MAX_QUEUES of 512 causes pretty high memory need for the application
>   (something between 1 and 2G) - is that really needed? I'm now running
>   with 32, and I'm able to get away with 256M. Can we tune this
>   default?

Don't think we need change default just because your platform is 32,
Well, my platform is 128, other platform may have other value, :-)

> 
> - hw_strip_crc is set to 0, but either the igb driver or the ET2 quad
>   port adapter I'm using is ignoring this. It does strip the CRC, so

Igb and ET2 should NOT ignore it.

>   does software, and I'm losing 4 bytes on each unpadded packet. Known
>   issue?
> 
> Thanks,
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate
> Competence Center Embedded Linux

Re: [dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[Jan Kiszka]

On 2015-08-08 02:39, Ouyang, Changchun wrote:
> 
> 
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jan Kiszka
>> Sent: Saturday, August 8, 2015 1:31 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] vhost-switch example: huge memory need and CRC off-
>> loading issue
>>
>> Hi again,
>>
>> two findings in the vhost-switch example code that can cause grey hair for
>> starters:
>>
>> - MAX_QUEUES of 512 causes pretty high memory need for the application
>>   (something between 1 and 2G) - is that really needed? I'm now running
>>   with 32, and I'm able to get away with 256M. Can we tune this
>>   default?
> 
> Don't think we need change default just because your platform is 32,
> Well, my platform is 128, other platform may have other value, :-)

Then let's make it configurable or explore the actual device needs
before allocating the buffer. The impact on memory consumption is way
too big to hard-code this, specifically as this is per physical port IIUC.

> 
>>
>> - hw_strip_crc is set to 0, but either the igb driver or the ET2 quad
>>   port adapter I'm using is ignoring this. It does strip the CRC, so
> 
> Igb and ET2 should NOT ignore it.

Well, they do not listen to us. What can I do to debug this?

According to eth_igb_rx_init, hw_strip_crc affects E1000_RCTL_SECRC in
the receiver control registers, and debugging confirms this (the bit is
unset in all E1000_RCTL accesses). But there is still no CRC at the end
of received packets. Possibly, there is some other knob that controls
CRC stripping?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[Thomas Monjalon]

2015-08-08 09:17, Jan Kiszka:
> On 2015-08-08 02:39, Ouyang, Changchun wrote:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jan Kiszka
> >> - MAX_QUEUES of 512 causes pretty high memory need for the application
> >>   (something between 1 and 2G) - is that really needed? I'm now running
> >>   with 32, and I'm able to get away with 256M. Can we tune this
> >>   default?
> > 
> > Don't think we need change default just because your platform is 32,
> > Well, my platform is 128, other platform may have other value, :-)
> 
> Then let's make it configurable or explore the actual device needs
> before allocating the buffer. The impact on memory consumption is way
> too big to hard-code this, specifically as this is per physical port IIUC.

You can add a run-time option and/or add a comment in the documentation.
It is just an example, so it must be simple to understand.

Re: [dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

[Ouyang, Changchun]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Sunday, August 9, 2015 4:56 PM
> To: Jan Kiszka
> Cc: dev@dpdk.org; Ouyang, Changchun
> Subject: Re: [dpdk-dev] vhost-switch example: huge memory need and CRC
> off-loading issue
> 
> 2015-08-08 09:17, Jan Kiszka:
> > On 2015-08-08 02:39, Ouyang, Changchun wrote:
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jan Kiszka
> > >> - MAX_QUEUES of 512 causes pretty high memory need for the
> application
> > >>   (something between 1 and 2G) - is that really needed? I'm now running
> > >>   with 32, and I'm able to get away with 256M. Can we tune this
> > >>   default?
> > >
> > > Don't think we need change default just because your platform is 32,
> > > Well, my platform is 128, other platform may have other value, :-)
> >
> > Then let's make it configurable or explore the actual device needs
> > before allocating the buffer. The impact on memory consumption is way
> > too big to hard-code this, specifically as this is per physical port IIUC.
> 
> You can add a run-time option and/or add a comment in the documentation.
> It is just an example, so it must be simple to understand.

Agree, that is better than changing the default macro value.