[dpdk-dev] [ovs-dev] Traffic scheduling by qos_sched library in DPDK

[<gayathri.manepalli@wipro.com>]

Hi Team,

I started working on implementing the QoS Shaping in OVS+DPDK by making use of rte_sched library provided in DPDK.  Meanwhile to compare the performance, started performance test with DPDK sample scheduling application. Below are the configuration details of system which I am using,

Server Type : Intel ATOM

Huge page configuration: (Each page of 2M size)

[root@ATOM qos_sched]# grep -i huge /proc/meminfo
AnonHugePages:     90112 kB
HugePages_Total:    7168
HugePages_Free:     7168
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Port Capacity : 1G (All four ports)
I am able to successfully run the qos_sched with three pfc's as below,

./build/qos_sched -c 0x3e -n 1 --socket-mem 14336 -- --pfc "0,1,2,3" --pfc "1,0,3,2" --pfc "2,3,4,5" --cfg ./profile.cfg

Issue :

When I am trying to add one more profile configuration flow(4th one) , I am getting below error

Command : ./build/qos_sched -c 0x3e -n 1 --socket-mem 14336 -- --pfc "0,1,2,3" --pfc "1,0,3,2" --pfc "2,3,4,5" --pfc "3,2,5,4"  --cfg ./profile.cfg

Error:

done:  Link Up - speed 1000 Mbps - full-duplex
SCHED: Low level config for pipe profile 0:
        Token bucket: period = 1, credits per period = 1, size = 100000
        Traffic classes: period = 5000000, credits per period = [5000000, 5000000, 5000000, 5000000]
       Traffic class 3 oversubscription: weight = 0
        WRR cost: [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]
SCHED: Low level config for subport 0:
        Token bucket: period = 1, credits per period = 1, size = 100000
        Traffic classes: period = 1250000, credits per period = [1250000, 1250000, 1250000, 1250000]
        Traffic class 3 oversubscription: wm min = 0, wm max = 0
EAL: Error - exiting with code: 1
  Cause: Cannot init mbuf pool for socket 3

Analysis:

I have analyzed DPDK source code to find the root cause. I could see that in qos_sched, memory allocation while creating each mbug pool (rte_mempool_create) for corresponding RX port is as below,


MBUF_SIZE = (1528 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)

mp_size  =  2*1024*1024



>From above I understood that, for each pfc/ rx port around 4G of huge pages are consumed. Whereas ATOM is capable of maximum 7168 huge pages of 2M which is 14336M in total. So I am not able to configure beyond three pfc's. But I would like to measure the performance with 4 port & 6 port scenario which requires 4-6 pfc's configured.



Is there any other alternative through which I can configure more number of pfc's with my system configuration provided above.



Thanks & Regards,

Gayathri

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com